Launching today
jitllm
An operating system for LLM inference
1 follower
An operating system for LLM inference
1 follower
One binary. No dependencies. Drop it in and run. Up to 4.62x faster prefill than llama.cpp and 5.53x faster batched decoding than vLLM. jitllm detects your hardware and optimizes kernels and model placement across CPUs and GPUs. Run open-weight flagships, even larger than memory. OpenAI- and Anthropic-compatible APIs, desktop and terminal apps. Linux, macOS and Windows.
