Cekura - The self-improvement loop for voice agents
by•
Cekura is the testing, observability, and self-improvement platform for production voice and chat AI agents. It simulates thousands of scenarios, catches failures, diagnoses the root cause, rewrites prompts and config, then re-validates with a full regression sweep. Unlike tools that hand failures back to your team, Cekura closes the loop by fixing the agent itself and proving the fix holds without overfitting.
Replies
the loop-closing part is the interesting claim. the failure mode i would worry about is the simulator quietly becoming the thing you optimise against, so the agent gets very good at passing your scenario set and no better in production.
we run agents against third-party systems we do not control, and the split that ended up mattering most was "the agent did the right thing" versus "the other side actually accepted it". those two look identical in a transcript and come apart constantly in practice.
curious how the regression sweep decides a fix generalised rather than fit the scenarios. is the held-out set drawn from real production calls, or generated the same way the training scenarios were?
Cekura
@whateverneveranywhere That’s exactly the failure mode we guard against. We separate “the agent attempted the right action” from “the external system accepted it” using provider state, tool results, and outcome signals—not transcripts alone.
The regression sweep includes the full validation set plus happy/edge cases, with overfitting checks for scenario-specific fixes. Production-derived held-out cases are the strongest validation.
Cekura
It's like living in the future
Daily.co
I'm a huge fan of the work the Cekura team is doing. Closing the development loop with testing, evaluation, and paths to verifiable improvement is the single most valuable thing you can do in an agentic coding context. This is hard for voice agent development, for a bunch of reasons, including that there are many moving parts, gathering metrics robustly can be tricky and subtle, and success criteria are complex. Cekura has created great building blocks and high-leverage agent skills/workflows for solving these problems. Congratulations to the team on this launch, and all the effort and good thinking that went into it!
Cekura
@kwindla Thank you so much! We really appreciate your help with this
the without overfitting part is doing a lot of heavy lifting in that description, and its the right thing to be worried about. an auto-fixer that cant prove the fix generalized is just prompt roulette.
Cekura
@alex_watson2110 Agreed , that's exactly the risk we designed against. Every applied edit passes through an Overfitting Gate before validation, then gets re-run against the full evaluator set, so a fix only counts once it holds broadly, not just on the case it was written for.
@dileepchagam late reply, sorry. that gate answers the per-fix version of the problem well.
the one it does not cover is the cumulative version: if the fixer keeps optimising against the same evaluator set, that set quietly becomes the training set, and you end up fitting the benchmark rather than the behaviour. worth holding a slice of evaluators back that the fixer never gets to see, and only checking them periodically. if the visible score and the held-back score start drifting apart, that is your tell.
MonoCloud for Startups
Congrats on the launch. The hard part of a self-improvement loop is the blast radius. When an agent rewrites its own behavior from production calls, a fix for one flow can quietly bend a compliance flow sitting next to it, and in healthcare that is the thing every security review hunts for. Closing that loop safely, so the agent improves without drifting out of its guardrails, is the whole game, and it is a genuinely hard problem to have taken on.
Cekura
@roguetink Thank you, really appreciate it
Cekura
Lets Go Team!!!!
Simulating messy real-world conversations with interruptions, pauses, and background noise feels much closer to production than traditional scripted evaluations.
Cekura
@pratham_patel8 Thanks, appreciate it
Love seeing the focus on proving that a fix doesn't introduce regressions elsewhere. Reliable AI systems need repeatable validation, not just faster debugging.
Love the idea of making voice agents “self-healing” instead of just observable.
Cekura
@abhi_nebhnani thanks! appreciate it.
this is squarely the problem we deal with running voice AI. the thread so far is all about logical correctness (did the agent do the right thing), but a huge chunk of our real failures are cases where the words are technically right and the delivery is off - wrong pacing after an interruption, a flat tone on something that should sound apologetic, talking over a caller who paused to think rather than finished. does Cekura's simulation/scoring catch prosody and delivery quality as a distinct failure category, or is it mainly evaluating on transcript content right now
Cekura
@galdayan Good question. We provide several out-of-the-box metrics for exactly this , pacing, interruption timing, and voice tone/clarity , computed straight from the audio. We also support custom metric creation, including audio-based ones, so you can define your own metric to judge delivery, tone, or pacing beyond what's built in
@dileepchagam good to know pacing and interruption timing are already built-in rather than something we'd have to define ourselves. the case I'm most interested in is the interruption-timing one specifically - talking over a caller who paused to think vs one who actually finished is a judgment call even for a human listening back, since the only signal is a gap length that could go either way. is that metric scored against a fixed silence threshold, or does it try to account for what's actually being said before the pause (trailing off mid-sentence vs a completed thought)
Cekura
@galdayan Hi Gal, regarding your use case for interruption timing during caller pauses: while we don't have a pre-defined metric for this, you can easily build a custom one. Our Metric Builder lets you describe what you need in plain text, and it generates the metric using your recent call data. Since custom metrics analyze both audio and transcripts, your scenario is easily supported. Also we have Metric optimiser feature too where you can use false positives calls to optimise the metric further. Hope this helps.
@rishabh_sanjay good to know it's buildable, but that puts the burden on me to define "completed thought" from scratch in plain text. since you already run transcript analysis alongside audio, does that layer surface anything like incomplete-utterance signals (trailing conjunctions, unfinished clauses) that I could point the Metric Builder at, or would I genuinely be starting from a blank page describing linguistic completeness myself