AI raised commits 240% and releases 30%. The gap is the only number that should change your Q4 plan.

by

A new NBER paper tracks 500,000+ GitHub developers against their actual AI telemetry. The task-level gains are enormous and they mostly don't survive the trip to a shipped release. Across four app marketplaces: a sharp rise in new apps, no rise in total usage.

Most of the AI-productivity argument you've read this year is two camps shouting single numbers at each other. There's a paper out that makes both camps right and is more useful than either, and the revised version went up this month, so it's worth twenty minutes before you write your Q4 plan.

It's NBER working paper 35275, Demirer, Musolff and Yang - "Writing Code vs. Shipping Code." More than 500,000 GitHub developers, matched against their actual AI usage telemetry rather than a survey, in a matched event study design.

The headline numbers are large and I have no interest in arguing with them. Cumulative effects on commits: 30% for autocomplete, 180% for interactive coding agents, 240% for autonomous ones. If someone tells you agents made them dramatically faster at writing code, this is the paper that says they're right.

Then read the next sentence of the abstract, which is the one nobody screenshots.

That 240% falls to 80% for the number of projects, and to 30% for actual releases.

Same developers, same period, same data. Three-quarters of the gain evaporates somewhere between the commit and the thing a user can install.

What the attenuation actually is.

The authors call it the weak-link hypothesis: production is a chain, and speeding up one link moves the output by however much that link was binding. They put an estimated elasticity of substitution of 0.23 between AI and human effort - which, if you haven't thought about elasticities since school, is the formal way of saying these are strong complements, not substitutes. Cheap AI effort does not replace the human effort around it. It makes the human effort the thing that determines your output.

So the interesting question stopped being "how much faster is the agent." It's "what is my weak link," and for most small teams that answer isn't code. It's deciding what to build. It's review. It's App Review. It's support, onboarding copy, the migration nobody wants to own, the decision that's been in a Slack thread for three weeks. None of that got 240% faster, and now all of it is fed by an upstream that did.

The practical version: if you add agents and your release cadence doesn't move, that isn't a tooling failure and buying a better agent won't fix it. You've just found your bottleneck, precisely, for free.

The finding that should matter most on this site.

The paper also checks four major software marketplaces, and finds a sharp increase in the number of new apps and no increase in total usage.

Read that twice if you're planning a launch. Supply of software went up. Demand for software did not. That is the entire competitive picture for anyone shipping an indie product right now, in one sentence, from an independent source rather than from a vibe.

It reframes the launch problem. If you've felt like launching got harder over the last two years while your product got better, this is the mechanism: you're not worse at this, the denominator grew. It also means distribution, not build speed, is the scarce thing, which is roughly the opposite of what the tooling spend of the last eighteen months assumed.

Who paid for this, and why I'm citing it anyway.

Two of the three authors previously held postdoctoral positions at Microsoft and now work as paid research consultants for the company. That's disclosed on the NBER page, and I'd flag it on principle, since I've spent other posts complaining about vendor-measured numbers.

Here the conflict runs backwards, which is why I trust it more rather than less. Microsoft sells Copilot. A paper finding that the gains largely fail to reach shipped software is not the paper you commission to sell seats. Take the disclosure seriously and note that it cuts against the result.

One number to stop quoting.

While you're auditing your priors: if you still cite METR's "AI made developers 19% slower," check the page. METR has marked that study out of date themselves.

Their follow-up is more interesting than the original. They ran it again from August 2025 with 57 developers, and published in February 2026 that the design had broken - not because of the result, but because too many developers refused to take part or quietly withheld their hardest tasks rather than do them without AI. Between 30% and 50% told the researchers directly that they were filtering which tasks they submitted. The raw estimate flipped to a speedup, with confidence intervals wide enough to drive a truck through, and METR's own read is that their number is a lower bound.

The study failing is the finding. Developers would not give up the tools for $50 an hour. That tells you something the 19% never did.

The exercise.

Open your own release history for the last six months, not your commit history. If commits are up and releases are flat, write down what sits between them, in order. That list is your Q4 roadmap, and almost nothing on it is a coding problem.

I did this for Murror and the honest answer was that our weak link is me - specifically the decision of what's worth building, which agents have made harder rather than easier, because the cost of building the wrong thing dropped and so did the friction that used to stop us.

Two caveats. It's a working paper, revised this month, and the marketplace analysis is the part I'd most want replicated by someone with no Microsoft relationship at all. And "releases" is a crude proxy for value - a team shipping fewer, larger releases isn't underperforming, it just looks that way in this data.

42 views

Add a comment

Replies

Best

I made the exercise. Commits up, releases flat, and the queue between them is written down in my repo as a rule, which makes it easy to read off.

A feature isn't shipped here until its docs and marketing surfaces ship in the same change: the product doc, the in-app docs page, the llms.txt builder, the FAQ of every platform it touches, and that platform's SEO file. 5 artefacts, none of them code.

However, my own rule is part of the bottleneck. I wrote it after the FAQ spent months recommending a tool for audio we already generate, so it's a brake I installed on purpose

I woulf be curious how Murror separates a genuine delivery bottleneck from a team simply choosing to ship fewer larger releases?