The Refactor Benchmark

The setup

We wrote a deliberately messy TypeScript codebase: an order system with 840 lines, 11 modules, and 67 tests. It carries the debt you find in legacy code. A duplicated pricing engine. Deep nesting. Magic numbers. Dead code. We copied the codebase into two identical repositories. The same coding agent, with the same model and the same prompt, refactored both. The MCP run could call the aipatterns MCP server and read its skills. The plain run had no server and no skills. One static analyzer scored both results after the runs. This was a small test: one run per condition. We say more about that below.

The numbers

What the numbers say

With the server, the agent removed nearly twice as much code: 100 lines against 53. It removed a third of the duplication. The plain run removed a fifth. The maintainability index rose in both runs. It rose a little more with the server. The run with the server did more cleanup and broke nothing. Every original test passed, and the typecheck reported zero errors in both runs.

How the agent used the server

The transcript shows 8 calls to 6 server tools. The agent asked for smell detection, read a smell treatment, read the refactoring steps, and ran a pattern-fit check. It also worked through 4 skills. The pattern repeats: look up the treatment, read the steps, then change the code.

This is a small test

Treat these numbers as a signal, not a proof. We ran each condition once. That is one data point per condition. And 840 lines is small. A small codebase caps the possible win. A modern model already knows how to fix a duplicated pricing engine at this size. It does not need help to find dead code in one file. The server can only add guidance the model does not already have. We expect a bigger effect on a bigger codebase. An agent cannot hold 20,000 lines in mind. Named smells, named treatments, and numbered steps then do real work: they show the agent the next move. A bigger benchmark across more models and agents is coming.

Give your agent the same help

The server and the skills are public. The /agents page shows the setup for your client. Then let the agent refactor your messiest module, and measure the result the same way.