Launching today
oqoqo
Build evals and custom benchmarks for real-world tasks
524 followers
Build evals and custom benchmarks for real-world tasks
524 followers
Run eval experiments at scale in realistic environments. Define custom task sets to build your private benchmarks, measure how well agents can use any product, and find best models for your use cases. Generate dynamic insights to detect frictions in product interfaces or token inefficiencies.
Free Options
Launch Team / Built With
Sulu
This is super useful! I’ve been building agents and skills to make product onboarding easier for enterprise customers but right now its kind of a black box - I don’t really know how they are using it, where things are breaking and what I should fix first. If I get to see how the agent behaves across diff scenarios and where users are getting stuck it will be huge. Can't wait to use the CLI and run this on autopilot!
oqoqo
@priyansh_rastogi yess! try out the plugin for claude code. It is so much fun to just let claude handle the setup and experiment runs. Also really helpful to create custom visualizations.
Serand
oqoqo
@rukhsar_amjad The criteria can be as elaborate/simple as you want, having criteria around quality definitely helps manage this.
TrackerJam
The token efficiency insights caught my attention. Small inefficiencies can become pretty expensive when agents run at scale.
oqoqo
@maklyen_may Definitely. We have found it really helpful to run multiple trials and see the trends, really brings the inefficiencies to the front.
Pictioner
Evaluating models and harnesses in an easy, consistent way is hard. Good to see your platform take up the challenge and ease the entire process. 10/10 recommend
oqoqo
@priyankar_kumar1 Thanks Priyankar!