Launching today
RepoGym
Validate the task before you benchmark the coding agent
1 follower
Validate the task before you benchmark the coding agent
1 follower
Turn repository tasks into repeatable coding-agent evaluations. Define a task, check that doing nothing fails and the reference fix passes, then inspect scores and trajectories. Includes Python and JavaScript examples, composable graders, task mining and HTML reports. Free, MIT-licensed Python library. Start from the source checkout; local execution is for trusted code.
