Local LLM: MITHRIL

Hi everyone my name is Josh! I do a lot of work developing with AI tools and workflows and lately in particular experimenting with local LLMs. I've spent a bit of time building this local LLM suite to gain some experience developing with models locally on iOS. There's so much to dive into... MLX, CoreML, llama.cpp, Executorch, quantization.... Mithril allows you to explore and begin experimenting with some of these models and frameworks right on your phone and even has some cool features: -option to choose inference engine Llama.cpp vs. Executorch -RAG chat for both in-chat conversation as well as upload of documents to chat against (local sqlite db allows for deletion & json export in-app) -Metal acceleration to take full advantage of iPhone -web search capability powered by duckduckgo (anonymous search) optional -speech-to-text in chat powered by Whisper cpp by Open AI -light 35mb install file I'm really enjoying developing this and I hope that some people find it interesting to use and even potentially helpful! Super open to continuing to build out new features so please suggest anything for next release! 100% free and available now on the App Store- I hope it works well for everyone! --- 📱 App Store: https://apps.apple.com/us/app/lo... 🌐 More info: https://mithril.solutions Deploy Forward: https://deployforward.com (my company focusing on developing software with LLM tools and workflows) Twitter: https://x.com/boshjerns | https://x.com/DeployForward Made possible by: • llama.cpp by Georgi Gerganov: https://github.com/ggerganov/lla... • llama.rn React Native bindings: https://github.com/mybigday/llam... • ExecuTorch PyTorch mobile inference: https://docs.pytorch.org/executo... •Huggingface and open-source community that continue to provide models, quantizations, techniques...