Launching today
CouncilAI
Local AI that runs 4 models and picks the best answer
5 followers
Local AI that runs 4 models and picks the best answer
5 followers
CouncilAI runs 4 AI models on your own hardware — completely offline, no data ever leaves your device. It routes questions by difficulty: simple ones go to a fast model, complex ones trigger a full deliberation where models compete and the strongest answer wins. v1.2 adds Sequential Mode (draft → critique → refine), Compare Mode, and a transparency panel showing exactly how every routing decision was made. Beta: $3 USD — councilai.ca
genuinely curious — what hardware are you realistically targeting for running all four models at once? even a decent quantized mix would chew through ram on most consumer machines, so wondering if there's a minimum spec you'd recommend
@zeliha8u0s Great question. CouncilAI doesn't run all four models simultaneously — that's an important distinction. The router (Llama3.2:1b) classifies the question, then a single responder model is loaded for that query. So at any given time you're running at most two models: the router and one responder.
In Council Mode all three responders do run, but sequentially via Ollama's model loading — not simultaneously in VRAM. Ollama handles the loading/unloading between them.
Minimum recommended specs:
8GB RAM — can run the full stack with quantized models (Gemma2:2b + Phi3:mini + Mistral q4)
16GB RAM — comfortable, no swapping issues
Dedicated GPU with 6GB+ VRAM — significantly faster responses
The hardware detection on first launch adjusts model selection to your specs — lower RAM machines get smaller quantizations automatically. On a 8GB machine with no GPU responses are slower (30-90 seconds for Mistral) but it does work.