Cekura - The self-improvement loop for voice agents
by•
Cekura is the testing, observability, and self-improvement platform for production voice and chat AI agents. It simulates thousands of scenarios, catches failures, diagnoses the root cause, rewrites prompts and config, then re-validates with a full regression sweep. Unlike tools that hand failures back to your team, Cekura closes the loop by fixing the agent itself and proving the fix holds without overfitting.
Replies
Cekura
@sunil_mallya Both , we use STT to build the transcript for content/accuracy grading, but for tone, pacing, clarity, and similar delivery signals, models analyze the actual audio stream directly rather than relying on the transcript.
Jots
Nice launch! What CI/CD tool integrations do you support?
Cekura
@avz Hi Julien, GitHub Actions is the one we support out of the box. For some other CI system there's no separate integration needed: our CLI does the same job. You can check more on CI/CD here: https://docs.cekura.ai/documentation/guides/github-actions-ci-cd
When monitoring production calls, how does Cekura detect quality issues in real time, and what kind of alerting or reporting does it provide to teams?
Cekura
@charlotte_henry For production monitoring, Cekura runs metrics continuously on live calls and clusters failures into recurring failure-mode themes automatically, so you see patterns instead of re-reading every call. On top of that, configurable alerts (failure, trend drift, threshold breach, new failure mode, failure-mode spike) route straight to Slack with the call and context attached.
Does Cekura support testing across multiple languages and accents, given how critical that is for conversational AI reliability?
Cekura
@eden_halls Yes , language and accent are both first-class testing surfaces. Personalities let you run the same evaluator across different languages (including code-switching, like Spanglish) and different regional/non-native accents, with a Transcription Accuracy metric that flags exactly where STT struggles per variant, so you can compare pass rates side by side instead of relying on one aggregate number.
How does Cekura evaluate more nuanced qualities like tone, empathy, or conversational flow, beyond just accuracy or task completion?
Cekura
@louis_hallie Yes , beyond accuracy/task-completion, we have built-in metrics for exactly that: CSAT and Sentiment score the caller's tone and satisfaction, Verbosity and Unnecessary Repetition flag conversational flow issues like over-explaining or re-confirming the same thing, and Voice Tone + Clarity looks at delivery quality from the audio itself. For anything more specific to your agent's context, you can also define a custom LLM-judge metric
For teams building voice based agents specifically, does Cekura account for latency and audio quality issues, or is the focus primarily on conversational logic?
Cekura
@sophie_louis Both, not just conversational logic. We have dedicated Conversation Quality metrics for latency (with percentile breakdowns) and interruption/turn-taking timing, plus Speech Quality metrics for audio itself , pitch, clarity, jitter, unnatural speaking-rate changes , all computed straight from the call audio
How customizable are the evaluation metrics? Can teams define their own quality benchmarks based on their specific use case?
Cekura
@william_leon1 Very customizable , beyond our pre-defined metrics, teams can define their own custom metrics, including LLM-judge and Python-based ones for fully custom logic tailored to their specific use case.
Does Cekura provide regression testing capabilities to catch quality drops when agents are updated or retrained?
Cekura
@james_khuzuma Yes , every prompt/config fix is re-run against the full evaluator set as a regression sweep before it's considered done, and the same evaluators can be run in CI/CD on every agent update so quality drops get caught before they reach production.
Thanks — the GitHub Actions path is the answer to that question. The follow-up I'd have is whether the infra regression suite runs against a live agent clone or replays recorded sessions, because customer-support agents with integration state tend to behave differently on replay versus a live environment.
Cekura
@hazy0 Hi Hazy, every Infrastructure Suite case is a real simulated call against a real running agent.
That is the answer to the question - replay-based evals have a way of missing failures that only show up with live state, so real simulated calls against a running agent is the right approach. Good to know the suite is doing actual call simulation rather than matching against recorded outputs.
Voice agents fail in ways text evals never catch — interruptions, latency, someone talking over the bot. The "loop" framing is the right one: testing voice once at build time is basically useless. Would love to know how many simulated calls it takes before the improvements show up.
Cekura
@lucasjpols Fair question , it's less about a fixed number of calls and more about iterations: the loop diagnoses, fixes, and re-validates until the full evaluator set passes. We've seen quite a lot of improvement starting from just 1-2 rounds itself