oqoqo
Build evals and custom benchmarks for real-world tasks
558 followers
Build evals and custom benchmarks for real-world tasks
558 followers
Run eval experiments at scale in realistic environments. Define custom task sets to build your private benchmarks, measure how well agents can use any product, and find best models for your use cases. Generate dynamic insights to detect frictions in product interfaces or token inefficiencies.
Free Options
Launch Team / Built With
CloudflareYour next app belongs on Cloudflare Workers. Deploy for free
Promoted
In the FAQ, you might consider changing "The free plan adds runs to your organization each month" to "The free plan resets runs for your organization each month" ("refreshes" could also work).
Adding runs, but runs not rolling over is slightly confusing.
Neat platform though. I can see how this would make it easier to do more sophisticated eval-ing without the hassle of building scaffolds and boilerplate from scratch.
@thedatadavis Aah these are the tiny details that do get missed out in the process of the launch. Thank you so much for pointing out the mismatch. Will be updated and fixed soon!
This is super useful! I’ve been building agents and skills to make product onboarding easier for enterprise customers but right now its kind of a black box - I don’t really know how they are using it, where things are breaking and what I should fix first. If I get to see how the agent behaves across diff scenarios and where users are getting stuck it will be huge. Can't wait to use the CLI and run this on autopilot!
@priyansh_rastogi yess! try out the plugin for claude code. It is so much fun to just let claude handle the setup and experiment runs. Also really helpful to create custom visualizations.
The idea of testing agents on the real world tasks make sense. How long does it take to setup an eval with oqoqo ?
@farhan_nazir55 congrats on your launch last day Farhan! you can hook up our plugin/MCP with your coding agent and allow the agent to create eval sets. This makes it really easy to get started. The hardest part of the process is making sure the instructions and rubrics that don't overexplain things to agents.
@rukhsar_amjad The criteria can be as elaborate/simple as you want, having criteria around quality definitely helps manage this.
Evaluating models and harnesses in an easy, consistent way is hard. Good to see your platform take up the challenge and ease the entire process. 10/10 recommend
@priyankar_kumar1 Thanks Priyankar!
Congrats on the launch. Oqoqo looks like a really interesting approach to evaluating AI agents in realistic environments.
@priyankamandal thanks priyanka
@istiakahmad thanks istiak!