All activity
Manuel Herrador Muñoz, PhDleft a comment
Sloth-like speed? More like a context beast. 🦥🚀 Most of us have hit that wall where a 6GB or 8GB GPU just gives up the ghost as soon as you feed it a long PDF. Seeing an implementation of Google's TurboQuant (ICLR 2026) this early is a game-changer for the local LLM scene. Why this is a big deal: - VRAM Magic: It’s basically "downloading more RAM" but for your GPU. Compressing the KV cache from...
QuanslothGUI Based on the implementation of Google's TurboQuant
Based on the implementation of Google's TurboQuant (ICLR 2026) — Quansloth brings elite KV cache compression to local LLM inference. Quansloth is a fully private, air-gapped AI server that runs massive context models natively on consumer hardware with ease - PacifAIst/Quansloth
QuanslothGUI Based on the implementation of Google's TurboQuant
Manuel Herrador Muñoz, PhDstarted a discussion
Based on Google's TurboQuant — Quansloth runs massive context models on consumer hardware with ease!
Hi there, I am new here! Sloth-like speed? More like a context beast. 🦥🚀 Most of us have hit that wall where a 6GB or 8GB GPU just gives up the ghost as soon as you feed it a long PDF. Quansloth (Apache 2.0 License at GitHub) an implementation of TurboQuant (ICLR 2026) this early is a game-changer for the local LLM scene. Why this is a big deal: VRAM Magic: It’s basically "downloading more RAM"...