RuleProbe
Verify whether AI coding agents actually follow your rules
Start new thread
Brad Kinnard

RuleProbe - Verify whether AI coding agents actually follow your rules

by•
RuleProbe reads the same instruction file your AI coding agent reads (CLAUDE.md, AGENTS.md, .cursorrules) and checks whether the agent's output actually followed the rules. AST analysis, filesystem checks, regex pattern matching. 53 matchers across 9 categories. Every failure includes file, line number, and what was found. No LLM in the verification pipeline. Deterministic, reproducible, ships as CLI and GitHub Action.

Add a comment

Replies

Best
Brad Kinnard
Maker
📌
RuleProbe exists because every AI coding agent reads instruction files and none of them prove they followed the rules. You write camelCase variables, no any types, named exports only, test file for every source file. The agent says "Done." Then you spend 20 minutes in code review manually checking what it missed. RuleProbe automates that check. It parses 6 instruction file formats, extracts rules that map to deterministic AST, filesystem, and regex checks, and verifies agent output against each one. Pass/fail with file paths and line numbers. Same input, same output, every run. v0.1.0 covers TypeScript and JavaScript with 38 matchers across 9 categories. Ships as a CLI (npx ruleprobe verify CLAUDE.md ./output), a programmatic API, and a GitHub Action that runs on PRs with no API keys beyond GITHUB_TOKEN. Optional LLM-assisted extraction for rules the static parser can't reach. User-defined rules via ruleprobe.config.ts for anything project-specific. Four runtime dependencies, all pinned to exact versions. MIT license. https://github.com/moonrunnerkc/...