Skill Bench

Automated evaluation for Claude Code skills.

2 followers

Automated evaluation for Claude Code skills.

2 followers

Automated evaluation for Claude Code skills. Grade responses with evidence, track regressions, and report results directly in pull requests.

Free

Launch tags:GitHub

Launch Team

Render Sandboxes: a computer for every agentRun untrusted code safely. Idle sandboxes cost nothing

Promoted

Maker

📌

Introducing Skill Bench - Automated testing for AI agent skills I built a GitHub Action that evaluates Claude Code skills against test cases with automated grading and PR reporting. What it does: - Runs eval cases via claude -p in CI - Grades responses with evidence-based criteria - Posts results directly on pull requests - Retries with backoff on API timeouts - Supports parallel evaluation of 30+ skills via matrix strategy Important: This action uses the Anthropic API with your own API key. Each eval case makes 2 API calls (execute + grade). Use the "changed only" workflow pattern to avoid evaluating unchanged skills on every PR. Monitor your usage at Anthropic console.

Report

8mo ago

Reviews