Launching today
Argmin AI
Test AI agent workflows without ML expertise
56 followers
Test AI agent workflows without ML expertise
56 followers
For teams and founders shipping agentic AI workflows and features. Generic metrics don't speak your product's language, so a new prompt, model, or RAG change can pass them and still ship real issues. A quick manual check catches even less. You don't need annotated data, an ML team, or a month to be safe from regression. Drop in your agent's task, business rules, docs, and examples, and Argmin AI builds an evaluation you run before every change.
Quorini
@dmitrii_konyrev Really well explained. Quick question: how heavy is the initial setup for a new agentic workflow?
Do we need to provide any initial examples or can Argmin bootstrap the evaluation metrics mostly from our description of success?
@alex_tartach Thanks Alex. Setup is light: Argmin bootstraps the eval and the metrics mostly from your description of success plus your docs, so you don't need to bring examples to get started. A handful helps with precision though, dropping in 10-20 (especially the failure cases you actually worry about) sharpens the judge and shortens calibration. One honest caveat for agentic specifically: the final-output eval bootstraps cleanly from the description, while step and trajectory-level checks are more hands-on right now. When you say a new agentic workflow, is "success" the final answer, or the whole path it took to get there?
@levington27 Straight answer: it doesn't keep pace on its own yet. As scope grows (more tools, longer chains) you regenerate from the updated task description and drop in a few of the new traces, and the judge has to be re-checked against your experts for the new behavior, because one calibrated on the old scope will quietly grade the new steps wrong. That refresh is a deliberate step today, not automatic. Making the eval track scope, especially at the trajectory level and not just the final output, is exactly the hard part we're building toward. Is your scope creep more new tools, or longer chains?
what caught my attention is the focus on evaluation data rather than the evaluator itself. in many ML projects, preparing a reliable evaluation set is where a considerable amount of time disappears. if this can simplify that step, it would be genuinely useful.
@anastasia_gromova1 You're pointing right at the time sink, Anastasia. Building a reliable eval set by hand is where whole sprints quietly disappear. One thing we ran into: the set and the evaluator are coupled. A clean eval set only matters if whatever grades it actually agrees with your experts, so we bootstrap both from your task and docs, then calibrate the judge against a handful of your own labels.
Quorini
Seems to be great tool for AI agentic at scale. Looking forward to give it a try! Is there relevant documentation or guide on how to start using it?
@alex_tartach Appreciate it Alex. Getting started is quick, you go from your task description to a first eval in a few minutes. Happy to send you the getting-started guide and help you get your first agentic eval running. Want me to DM it over with a couple of tips for the agentic setup?
love the direction. most teams don’t have perfectly labeled datasets, and building them from scratch usually turns into a project of its own. starting from production traces instead feels much closer to how things actually work. gonna give it a try with one of our internal workflows and see how it does.
@dmitrii_manuev Appreciate it Dmitrii. If you're starting from traces, pick 15-20 that include a couple of the failures you actually worry about, not just the clean happy-path ones, that's what makes the generated eval catch something real on the first run.
Would love a way to compare eval runs side by side so we can see exactly where a new prompt version regressed before merging. A diff view across runs would save a lot of guesswork.
@yaz2029848 This is on the money, Yağız. Catching the regression before you merge is exactly where evals earn their keep, and the part that makes it actionable is a per-example diff across versions, not just an aggregate score nudging up or down. It's high on our list. One question: when a run regresses, do you care more about the aggregate delta, or about seeing the specific examples that flipped? That changes how we'd build the diff.
the fact that it generates evals from just a task description and a few examples is genuinely impressive. most eval tools assume you already have a labeled dataset sitting around, so skipping that whole step is a real unlock for smaller teams shipping fast.
@esma144191 Thanks Esma. The labeled-dataset assumption always felt backwards to me: the feature you most need to eval is the new one, which is exactly the one you have no labels for yet. Starting from the task description plus a couple of examples gets you a first eval on day one, then you tighten it as real cases come in. Anything you're shipping right now where that would've saved you?