KubeAI: Private Open AI on K8s
Serve LLMs privately with an OpenAI API compatible API
20 followers
Serve LLMs privately with an OpenAI API compatible API
20 followers
ā
ļø Drop-in replacement for OpenAI with API compatibility š Serve OSS LLMs on CPUs or GPUs āļø Autoscaling with scale from 0 š ļø Zero dependencies (no Istio, Knative, etc.) š¤ Operates OSS model servers (vLLM and Ollama) š Chat UI included
KubeAI: Private Open AI on K8s launches
Launch date
KubeAI: Private Open AI on K8sServe LLMs privately with an OpenAI API compatible API
Launched on August 28th, 2024