Launching today
Colloqio
On-device AI - private, fast, always available
55 followers
On-device AI - private, fast, always available
55 followers
Meet Colloqio, the revolutionary AI companion that runs 100% on your device. No cloud servers, no data collection, complete privacy. Your personal AI that remembers you, works offline, and keeps your conversations truly private. Coming soon to iOS.
Colloqio
@invalidrun Hi Daniel, I love this. how did you get it to work technically?
Colloqio
@zolani_matebese Thanks for the question! Under the hood the model running is Gemma 3n, I did test various other models before settling on Gemma 3n and this model came out on top in terms of what I wanted Colloqio to do, memory usage, speed and how up to date the information was that it could reference. Then built the chat system and noticed quite quickly there were some limitations around long conversions, I would get maybe 12-15 long messages in and was running into issues so I implemented a system that basically compacts and summarises the current chat whilst it's the users turn to reply and then feed that back into the AI to carry on the conversion, I then implemented "Memories" basically the AI is told to pick out what it thinks might be important conversations to remember and then keeps them in memory for future conversions. Those were the two big hurdles I struggled with but everything else was relatively straight forward, there maybe still some issues with context and long conversations but I'm currently working through them as they arise.
Thank you for the question and I hope this gives you a brief insight into the technical parts of the app.
Really cool! What’s the biggest limitation users should expect from offline models?
Colloqio
@lak7 Hey thanks for the comment! In terms of limitations I would say it works well for simple tasks, chatting, braining storming ideas, asking questions but if you try to get into complex chats and start getting into more detail of complex tasks it just isn't built for that if you use it for drafting messages, asking questions about a subject, home work (not complex homework) it's fine but go beyond that and you will see some limitations, initially I also had issues with long conversations but I've managed to get around some of those issues (not all) by compacting similar to other LLM's do when the context gets too long.
@invalidrun I see! Pretty cool man, I do have a lot of use cases for such simple and offline tasks. Currently I am using runanywhere as an offline ai, but Colloqio's ui is definitely seems better, hope it soon comes to android!
Congrats on the launch on-device AI is a strong angle. I noticed one spot where first-time users might pause around setup/expectations. Happy to share a concrete example if useful.
Colloqio
Better if it will use voice commands only and talk to you like a support person.