Launching today
Argmin AI
Test AI agent workflows without ML expertise
58 followers
Test AI agent workflows without ML expertise
58 followers
For teams and founders shipping agentic AI workflows and features. Generic metrics don't speak your product's language, so a new prompt, model, or RAG change can pass them and still ship real issues. A quick manual check catches even less. You don't need annotated data, an ML team, or a month to be safe from regression. Drop in your agent's task, business rules, docs, and examples, and Argmin AI builds an evaluation you run before every change.
Finally an eval tool that gets agentic workflows without forcing us to label datasets. Dropped in our task description and a few examples and got a usable eval suite the same day.
@hayrettin988209 This is the one I'll be re-reading all day, thank you Hayrettin. Same-day usable was exactly the bar we set for ourselves. The real test now: run it for a week and see if it catches a regression you'd otherwise have shipped, that's when it earns its place. What kind of agentic workflow did you point it at, and did anything in the generated suite surprise you?
would love to see a github action or CI hook that runs the eval automatically on every PR and comments the diff in the checks section, that would make it actually part of the loop without anyone having to remember to trigger it
@berke9f56 This is exactly where we want it to live, Berke. An eval that only runs when someone remembers isn't really a gate, so a PR-triggered run that posts the diff into the checks section is the right shape. Honest status: it's on the roadmap, not shipped yet. Design question while you're here, do you want it to just comment the diff, or block the merge on a regression past a threshold? Those are two very different levels of trust in the eval, and I'm curious where teams want to start.
the way it builds evals from your own task description and docs instead of forcing you to hand-label data is a really thoughtful move for shipping agent features fast.
@adnancwzc Thanks Adnan. That was the whole bet: hand-labeling is where most teams stall out and quietly give up on evals, so if we can bootstrap from what you already wrote (the task description, the docs), you get to a real eval before you ship instead of three sprints later. What are you building agents for?
LLMOps.Space
You don't know what you don't know. If you're building agentic AI workflows and don't have a robust eval system, you're flying blind. This is a really cool solution.
@david_arakelian Flying blind is the right phrase, David. The scary part isn't the failures you catch in the demo, it's the confident-wrong outputs that look fine and ship anyway. That's the whole reason to have an eval you trust before you're in prod, not after. Appreciate you saying so, especially from someone living in LLMOps day to day.
love how you cut straight to the eval problem instead of padding with generic AI agent dashboards. the "drop in your docs and get evals" angle feels like exactly what shipping teams have been cobbling together with brittle scripts for too long.
@nazls86733 You nailed why we built it, Nazlı. The brittle-scripts phase is real: everyone reinvents a fragile grader, it slowly drifts from what their experts would actually say, and then nobody trusts the number. We'd rather own that one hard part, a judge that agrees with your domain experts, than ship another dashboard. What were you stitching together before?