AIR - Catch what your AI coding agent broke before it ships

by•
AIR records how your service behaves today: HTTP responses, database writes, messages, outbound calls, ordering and failures. Then you or any coding agent can refactor, upgrade or rewrite it. Run air protect after each change. It prints PASS, or exactly what changed, expected vs actual. Approve intended changes one scenario at a time. Works with Claude Code, Codex, any agent via MCP, and in CI. Free and open source (Apache-2.0).

Add a comment

Replies

Best

Hey everyone,

I use coding agents for a lot of my backend work now. What kept biting me wasn't code that broke. It was code that looked fine, passed the tests, and still did something different. A total calculated another way. An event with one item missing. A status code that changed.

So I built AIR. You point it at your running service once and it saves how the service behaves. After that, whenever you or an agent changes something, you run air protect. It says PASS, or it shows the exact field that changed with the old and new value. If the change was on purpose, you approve that one scenario and move on.

It looks at more than HTTP responses: database writes, messages, outbound calls, the order things happen in, and failures. There's no proxy and nothing to add to your service code. It works with Claude Code, Codex and any agent that speaks MCP.

I also ran a small benchmark. Same model, same migration tasks. Without AIR it got 0 of 9 right, with AIR's feedback 9 of 9. That's one model and one suite, so take it for what it is. The method is in the repo.

To try it:

npm install --global /air

air protect --url

Free and open source, Apache-2.0.

One thing I'd love to hear: what kind of change would you never let an agent make without checking it yourself? That's what I want AIR to cover next.

This is the failure mode I worry about most as a solo dev — not a crash, but a total or a status quietly changing while the tests still pass. Curious how you handle the noisy bits when you record behavior: timestamps, generated IDs, that kind of thing. Do those get normalized out, or do they show up as diffs you have to approve each time?

 That's exactly the case I built it for. A changed total or status still returns 200, so most tests never notice. For noise, AIR runs every new baseline twice against the legacy service. If a generated ID, timestamp or JWT differs between those two runs, it's checked by type instead of value: it must still be a valid UUID or UTC timestamp. Anything that came out the same both times, like a total or a status, stays an exact match. That way the noise drops out without hiding the values you care about. Typed tokens can also be bound by name. If the order ID in the response has to be the same one in the DB row, the published message and the outbound call, AIR checks that link on both sides, even though the old and new code generate different IDs. Some content is random by design, like ads or recommendations. For that AIR suggests a shape rule, which keeps the keys and types but not the values, and you add it with a written reason. Ignoring a field also needs a reason. There are no regexes, scripts or "90% similar" matching, because an ignored value is an unproven value. So you don't approve the same diff every time. You only see real changes, and air check approve --reason "..." records the ones you meant to make.