Cekura - The self-improvement loop for voice agents

Cekura is the testing, observability, and self-improvement platform for production voice and chat AI agents. It simulates thousands of scenarios, catches failures, diagnoses the root cause, rewrites prompts and config, then re-validates with a full regression sweep. Unlike tools that hand failures back to your team, Cekura closes the loop by fixing the agent itself and proving the fix holds without overfitting.

Add a comment

Replies

Best

How do you balance simulated conversations with real production data when improving evaluation quality? I'm curious how teams typically decide when one is more valuable than the other.

Closing the loop by fixing the agent and re-validating is the right shape, and the part I would want to know most about is what the loop is allowed to treat as ground truth, because for voice that is harder than the transcript makes it look.

We run an AI that answers and places real phone calls. We had a summariser that told users how a call went, and it fabricated confirmations on roughly a quarter of calls. The canonical one was "she picked up and the surprise was delivered successfully" on a call that lasted six seconds and was never answered. What did not fix it: lowering temperature, and putting "be honest" in the prompt. What did: forcing every claim to point at something literally present in the transcript, and naming the exact phrases it was forbidden to emit. That took us from about 24% to about 2% across roughly 1,200 calls.

The thing I would push on for a self-improvement loop specifically: the failures that hurt most in voice are often not content failures, they are timing failures, and a transcript-derived signal is blind to all of them. Latency before the first token, barge-in handling, a beat of silence at the top that reads as a dead line. We had calls whose transcripts were flawless and whose recipients hung up in the first few seconds, and nothing in the text said anything was wrong. If the improvement signal is derived mostly from transcripts, the loop will keep sharpening the part that already worked.

The overfitting claim is the other half of the same worry, and it is the harder one to prove. A regression sweep drawn from the same scenario generator that produced the fix is a closed system. Does the sweep draw from held-out production traffic, or from simulation?

What has been the hardest aspect of evaluating conversational quality that isn't obvious from metrics like latency or task completion?

Congrats to the team! Testing a voice agent before launch is one thing. Having it get better from its own real calls is a different level entirely. Closing that loop is the part everyone talks about and almost nobody ships.

closing the loop without drift is the real hard part 👏 how do you keep it from overfitting to one bad call?