Cekura - The self-improvement loop for voice agents
by•
Cekura is the testing, observability, and self-improvement platform for production voice and chat AI agents. It simulates thousands of scenarios, catches failures, diagnoses the root cause, rewrites prompts and config, then re-validates with a full regression sweep. Unlike tools that hand failures back to your team, Cekura closes the loop by fixing the agent itself and proving the fix holds without overfitting.
Replies
Congrats on the launch! Really like that you’re focusing on proving fixes instead of just identifying failures. One thing I’m curious about: how do you decide when a suggested fix is reliable enough to recommend versus flagging it for manual review?
Cekura
@ujjval_goury Hi Ujjval the loop run a suggested fix multiple times to see if the fix consistently resolves the failure mode. If the loop isn't sure about anything then it hands to the user for manual review.
Love seeing more attention on production reliability for voice AI. The regression validation especially stood out. Curious whether Cekura can prioritize issues by business impact (for example, failed payments vs. minor conversation hiccups) or if everything is treated equally.
Cekura
@yash_jain49 Great question. You can easily prioritize issues on which to self-improve your agent on Cekura. You can create custom metrics depending on your use case and can attach them to scenarios on which you are improving your agent and the self-improvement loop will consider the failure points on the attached metrics to improve your agent upon.
Love that this closes the loop instead of dumping another list of flagged calls on a human. The rule that it has to reproduce a bug before it is allowed to fix it feels very sane. For a team running a voice agent in something sensitive like healthcare intake, can you control which kinds of fixes it applies on its own and which ones need sign-off first?
Cekura
@doganakbulut Hi Dogan, for every issue the loop suggests a fix and verifies the fix across multiple runs to see if it is consistent but every change today requires your approval before it is applied. We are already working on adding a control which can enable the loop to automatically apply fixes for issues that are not critical and don't require manual review.
Cekura
Let's go team !! So excited for what's ahead.
Cekura
Let's go!!
Sourclip
Voice AI quality is surprisingly difficult to measure consistently. I like that this goes beyond transcript evaluation and looks at things like interruptions, latency, and delivery quality. That's usually where real production issues show up
HarnessRouter
@satvik_dixit Congratulations. And happy product launch.
Cekura
@huisong_li Thanks brother!
How do you balance simulated conversations with real production data when improving evaluation quality? I'm curious how teams typically decide when one is more valuable than the other.
Closing the loop by fixing the agent and re-validating is the right shape, and the part I would want to know most about is what the loop is allowed to treat as ground truth, because for voice that is harder than the transcript makes it look.
We run an AI that answers and places real phone calls. We had a summariser that told users how a call went, and it fabricated confirmations on roughly a quarter of calls. The canonical one was "she picked up and the surprise was delivered successfully" on a call that lasted six seconds and was never answered. What did not fix it: lowering temperature, and putting "be honest" in the prompt. What did: forcing every claim to point at something literally present in the transcript, and naming the exact phrases it was forbidden to emit. That took us from about 24% to about 2% across roughly 1,200 calls.
The thing I would push on for a self-improvement loop specifically: the failures that hurt most in voice are often not content failures, they are timing failures, and a transcript-derived signal is blind to all of them. Latency before the first token, barge-in handling, a beat of silence at the top that reads as a dead line. We had calls whose transcripts were flawless and whose recipients hung up in the first few seconds, and nothing in the text said anything was wrong. If the improvement signal is derived mostly from transcripts, the loop will keep sharpening the part that already worked.
The overfitting claim is the other half of the same worry, and it is the harder one to prove. A regression sweep drawn from the same scenario generator that produced the fix is a closed system. Does the sweep draw from held-out production traffic, or from simulation?
What has been the hardest aspect of evaluating conversational quality that isn't obvious from metrics like latency or task completion?