✅️ Drop-in replacement for OpenAI with API compatibility 🚀 Serve OSS LLMs on CPUs or GPUs ⚖️ Autoscaling with scale from 0 🛠️ Zero dependencies (no Istio, Knative, etc.) 🤖 Operates OSS model servers (vLLM and Ollama) 🔋 Chat UI included
You can try it on your laptop or your favorite K8s distro. Feedback greatly appreciated. Please share with others that you think may benefit from this. Building this was hard, but letting others know about it is harder, so your help would be greatly appreciated!
Report
No reviews yetBe the first to leave a review for KubeAI: Private Open AI on K8s
Websu