Launching today
LLM Test Server
Your LLM tests, without the provider
39 followers
Your LLM tests, without the provider
39 followers
LLM Test Server is a local, deterministic stand-in for 13 LLM provider APIs. Your app keeps its official SDK and points it at namespaced routes like /openai/v1; answers, tool calls, streams and failures come from fixtures your test sets up. The dashboard shows where every token goes: system prompt, tool schemas, conversation turns, prompt-cache reads and writes. Built forNET and Aspire, and shipped as a Docker image so any stack can use it. Free and open source.
Hi Product Hunt, I'm Konstantin, and I build LLM Test Server so you don't have to.
I found that a lot of the bugs in AI API are about incorrect api usage. The application sent the wrong request, dropped a tool result, stopped reading a stream too early, or retried an error badly. Those are ordinary integration bugs, and I wanted to test them the ordinary way, so ai agetns can fix it, before deploy. So same input, same result, for every run, with no live model in the loop.
LLM Test Server is a local server that speaks the HTTP APIs of 13 LLM providers, each under its own namespace such as /openai/v1 or /anthropic/v1.
Your app keeps its official SDK , just provide our base URL from the test server.
Of course you can setup script and simulate any api behaivor you may need for tests.
The part I like most is chaching oversvation. Becase are you sure your conversation collected correctly and did not break the cache?
For every request the we collect data and show it in the dashboard.
So what the context is made of (system prompt, tool schemas, user and assistant turns, tool results) and how much of it the prompt hit the cache.
It is for every stack. On .NET you get NuGet packages, an Aspire integration and a typed client. Everywhere else you run one Docker image, configure it over HTTP and point your Python, TypeScript or Go SDK at it.
The site is https://llmtest.managed-code.com
I would love to hear which LLM integration failure is hardest for you to reproduce today.
I don’t write the tests, so here is the view from the other side.
Our AI builds kept failing for reasons that had nothing to do with our code. A provider rate-limits CI, someone spends the morning proving it wasn’t them, the next run passes on its own. Nothing fixed, nothing learned, one morning gone.
That cost never shows up in any report, which is why it runs for months before anyone fixes it. This is the version of us that stopped doing that.
Has anyone here actually measured what flaky CI costs your team? Everyone I ask has a feeling about it and nobody has a number.
@alina_kostiuk_ 🔥
Hi! Good luck today! Nice to see a launch that solves a boring but very real problem.
@alexkhlystova thinks a lot ❤️
@alexkhlystova thank you!