SkillCI catches when a Claude model update silently breaks your Skill trigger stops firing, an instruction gets ignored, a redteam attack succeeds. Instead of just failing CI, it writes the failure as a permanent eval case you review and accept: catch once, covered forever. Also: OWASP-mapped local security lint, git-native bisect to find the breaking commit, and cross-model regression testing. Free, open source, single Go binary.

Hey PH 👋 Made this because I kept shipping Claude Skills that worked fine until Anthropic shipped a new model, and I'd only find out something broke by accident. The thing I'm most proud of is the self-growing eval loop — a caught regression becomes a permanent test automatically, instead of a report you read once and forget. I haven't found another eval tool that does that for either red-teaming or fuzz-testing. Honest gap: promptfoo has 50+ red-team attack plugins to my 16, and years of production hardening mine hasn't had yet. Not claiming to beat it outright — just that nothing else combines skill-aware security scanning + cross-model regression + git bisect + the self-growing loop in one tool. Repo has a demo GIF near the top showing the actual catch → generate → accept loop. Would love feedback, especially from anyone running Claude Skills in production already.

Ran it on a side project and the bisect feature actually pinpointed the exact commit where my tool calls stopped triggering. Loved that it wrote the failure as a permanent eval case instead of just nagging me with a red CI.

@wendell_hobson Thanks! That's exact goal, bisect finds root commit, locks it in as eval case so regression can't sneak back silent. Glad it caught real bug for you. Happy to get a feedback on other features if you have tried any?

SkillCI

Regression testing for Claude Skills

Regression testing for Claude Skills