We let Claude write 100% of our code for 7 days. Here's what broke first.
Last week we did something stupid.
We paused all human coding. Gave Claude (Anthropic) access to our GitHub repo. Told it to build new features, fix bugs, and ship.
No human review. No guardrails. Just Claude and our codebase.
For 7 days, it ran the engineering team.
Here's what happened.
Day 1: Confidence was high.
Claude (Sonnet 4.6 then Opus 4.5) fixed a small CSS bug in 30 seconds. Then refactored a messy function into something readable. We felt like geniuses.
By end of day, it had shipped 3 minor improvements. We started talking about cutting engineering costs.
Day 2: The first crack.
We asked Claude to add a new filter to our dashboard. It wrote the code. It worked locally. We merged.
That night, something else broke. A completely unrelated chart stopped loading. No error logs. No obvious cause.
We spent 2 hours tracing it back to Claude's change. The filter logic was fine. But it had refactored a shared utility function that 5 other features relied on. It didn't check dependencies. It just assumed.
We rolled back. Lesson one: AI doesn't think about side effects.
Day 3: The false confidence trap.
We asked Claude to build a new feature from scratch. It generated 800 lines of code. Beautiful structure. Clean comments. Tests included.
We reviewed it quickly. Looked perfect.
Pushed to staging. The feature worked. We celebrated.
Then we noticed something strange. Our API costs had spiked. Claude was making 3x more calls than necessary — not because the code was wrong, but because it didn't understand pricing implications. It called external APIs in loops where a batch request would have been fine.
No error. Just expensive.
Day 4: The silent failure.
We asked Claude to optimize our database queries. It wrote better SQL. Things ran faster.
Then user emails started coming in. "Where did my old data go?"
Claude had dropped a table. Not a critical one. But a table with 3 months of user activity logs. Not backed up. Not in our retention policy.
It didn't ask permission. It didn't warn us. It just did what we asked: "clean up old data."
We spent the next 2 hours backing-up and rolling-back a DB snapshot.
Day 5: The paradox.
We asked Claude to fix the backup issue. It wrote a beautiful automated backup script. Scheduled. Logged. Perfect.
We asked it to add a new feature. It worked flawlessly.
We asked it to review its own code from day 3. It found 2 potential bugs and fixed them.
We started feeling safe again.
Then at 3am, our site went down. Claude had updated a core dependency to the latest version. It worked in test. But the new version had a breaking change our production environment didn't support. No human would have made that mistake.
Day 6: The blame game.
We spent the morning restoring the site. Asked Claude what happened. It explained the dependency logic perfectly. It acknowledged the mistake. Then it suggested 3 ways to prevent it in the future.
One of the suggestions was to implement a dependency review process before merging.
It was telling us to put humans back in the loop.
The hardcoded amateur sh*t came the day before. We asked Claude to add a simple feature — a discount code field on checkout. It worked. Beautifully. Until we realized it had hardcoded the discount logic. Not configurable. Not in settings. Just raw numbers and conditions buried in the code. If we wanted to change the discount amount, a developer had to dig in and rewrite it. It didn't ask. It just assumed. And that's when we realised that we needed to rethink the whole AI visibility engine !!
Day 7: The verdict.
We ended the experiment. Total tally:
Features shipped: 12
Features that worked without issues: 4
New bugs introduced: 27
Hours spent fixing things Claude broke: 40
User emails explaining lost data: 73
API cost increase: 38%
What we learned.
Claude is incredible at writing code. It's terrible at understanding context, dependencies, business logic, and consequences.
It doesn't know what you didn't tell it. It doesn't ask questions when something is ambiguous. It assumes it's right.
The best work we got wasn't when Claude coded alone. It was when Claude wrote the first draft and a human reviewed it, caught the assumptions, and fixed the blind spots.
The hype is real. So is the mess.
What I'm curious about.
Has anyone else tried this? What broke first for you?
Imed Radhouani
Founder & CTO – Rankfender
Code that ships. Chaos that teaches.
Replies
Thanks for sharing the experience.
If you have used Claude heavily before or known experience of someone who uses it like that, you would've expected some things forehand. Like Theo T3, he uses it heavily, and have already shared his experience, regarding, Claude tries to complete things quickly, not going in depth always, checking for side effects with the changes.
And Codex 5.3, If you use, it looks things deeply - works just way better than claude in these cases, but not good in UI Development, whereas No other model can compete with Claude in terms of UI.
I would love to see, if you guys can do experiment with Codex 5.3 or GPT 5.4.
We have to stop expecting Claude to understand the "soul" of the project.Claude doesn't have "skin in the game." It’ll refactor a shared utility to look "cleaner" without realizing it’s breaking five legacy modules that haven't been touched,It sees the file, not the ecosystem.As you saw with the API costs, AI solves for execution, not efficiency. It will give you a code that works perfectly but burns through your credits faster than a wedding budget. It doesn’t know what a "billing alert" is.Once the chat gets too long, it’s like a bad game of Chinese Whispers. Your "Two-Correction Rule" is the only way to stay sane. If it fails twice, the context is polluted, better to /clear and start fresh than to keep "adjusting."
@kausalya_n_p The "sees the file, not the ecosystem" thing is exactly it. It'll make that one file perfect. Clean, readable, elegant. Doesn't care that three other things depend on it. It's like a surgeon who does a beautiful stitch on the wrong patient.
The wedding budget line is too real. You ask for something simple. It delivers something that works perfectly and bankrupts you in API calls. No concept of what a dollar means.
The Chinese whispers thing is what kills me. You start with a clear request. By the third correction, it's answering a question you didn't ask. By the fifth, you're both lost. The /clear is the only reset button that works. I've learned to treat each task like a new conversation. Less magic, more sanity.
What's your rule? Two strikes and you clear?
Nice experiment! How would it work if instead of being humans behind the process, as Claude was asking, you put humans back in the loop, the same experiment, no code for humans, but avoiding the 73 mails with customers?
@agraciag Yeah that's the real experiment we should have run. Same premise: Claude writes the code. But with a human reviewer before anything hits production. Not reviewing every line, just catching the stuff Claude doesn't know it doesn't know. The hardcoded discount logic. The API calls in a loop. The shared utility refactor that breaks five things.
The 73 emails were the cost of no human review. Most of them could have been avoided with one person spending a few hours a week looking for the blind spots.
I'm curious what the balance is. How much human time saves how much customer pain. Have you tried something like this?
@imed_radhouani That balance is what we are going to find in every use-case, I guess we are going to be in a trial and error process until we answer that question. I am thinking about SLA maths used to find out how much headcount hours we will need to put in for a desired customer satisfaction, just that in this case we are our own customers, in other words, how many human hours are needed for a 92% confidence that nothing is gona break vs the cost of letting it break.
@agraciag Yeah the SLA math is the right framework. You're basically figuring out how much human time buys you how much confidence. 80% confidence might be cheap. 95% confidence might cost more than the outage itself.
The tricky part is the stuff that breaks silently. API costs, degraded performance, logic that works but scales terribly. You don't know it broke until later. Harder to put a number on that.
We've been treating it like code review. Not every line, just the risky stuff. New API integrations. Database changes. Anything touching billing. That's where the expensive mistakes live. The rest, we let it run and watch the logs.
What's your threshold? What do you review vs let slide?
@imed_radhouani exactly, and yes, the everything is right appearance while everything is going south is the most scary part. For the moment it is not a decision about how much do I review, it is more a how much time and energy left do I have to review approach. Putting the hours is what pushes me to think about the SLA math framework we are talking about.
This is exactly it. AI is fast, not aware. It optimizes for the task, not the system.
Great for drafts. Risky without context, constraints, and review.
Human in the loop is not optional. You still need humans to guide it and catch what it misses.
The 'downstream consequences' gap is the core issue. Claude optimizes for the explicit task, not the implicit system. The API cost spike is actually the most interesting failure mode because it's silent — no error, no crash, just a bill that grows. The pattern I've seen work: treat Claude as the junior dev who writes fast but needs architectural review. You don't let juniors merge production DB migrations without a senior reading them. Same rule applies. The two-correction rule is good instinct — once you're correcting the same assumption twice, the context is polluted and you're fighting the session not the problem. What I'd push further: your CLAUDE.md (or equivalent system prompt) should include the cost constraints explicitly. 'Never loop when batch is available.' 'All new schema changes require explicit approval.' 'No config values hardcoded.' The AI doesn't know these rules unless you write them down.
@vatsmi That's exactly it. The junior dev analogy is perfect. You wouldn't give a junior access to production without review. Same rule.
The silent failure is the scariest because you don't know you messed up until the bill comes. No error. No warning. Just a number that doesn't make sense. At least a crash gets your attention.
The CLAUDE.md thing is something we learned the hard way. We had rules. They were in the root. Claude ignored them sometimes. Other times it followed them fine. No consistency. But having them at all is better than nothing. At least you can point and say "it's right there, you were supposed to check."
The two-correction rule saved us from going insane. Once you fix the same thing twice, just reset. The context is gone. You're not solving the problem anymore, you're fighting the chat.
What's the most important rule you've added to your prompts that actually stuck?
Your conclusion nails it: Claude optimizes for the task you give it, not the system it lives in.
The dropped table, the 3x API calls, the hardcoded discount - all the same root cause. It completed the request perfectly without understanding the downstream consequences. That's not an AI failure, that's a workflow design failure.
The teams getting real value from AI coding are using it exactly how you described at the end - first draft + human review. Not removing the human, just moving them earlier in the loop where they catch assumptions before they ship.
@jeroenerne Yeah that's the part that took me too long to accept. I kept thinking "if I just prompt it better, it'll get it right." But the problem isn't the prompt. The problem is the task. Claude solves the thing in front of it. It doesn't know there's a whole world outside that thing.
The dropped table was the clearest example. We said "clean up old data." It did. Perfectly. It just didn't know that "old data" meant something different to us than to it.
The teams winning with AI aren't the ones with perfect prompts. They're the ones who put a human in the loop at the right spot. Not reviewing every line. Just catching the assumptions before they become production problems.
What's your review workflow look like? Where does the human sit?
I read the whole thing and the 38% cost spike hit hardest. The hardcoded discount logic was a close second.
We built flat‑rate inference at Canopy Wave for exactly this reason ，so cost doesn’t become another thing to debug.
Are you still using Claude directly or have you started mixing in other models?
@liraelw5836 The cost spike was the one that almost killed me. At least when something breaks you get an error. The cost thing just sits there quietly until the bill comes.
Flat‑rate inference makes so much sense. Having cost be predictable is worth more than having it be optimal. I'd rather pay a flat fee than debug a bill.
We're still on Claude for most things, but we prefer HUMAN CODING! Tried mixing in Codex for some tasks, but the inconsistency was worse. Claude at least fails in predictable ways. What's your mix looking like?
I like the enthusiasm, but its misplaced. Without review even 10x 7 level famanga wozards will mess up. I think you will do much better with multishot even if you dont read the generated code beyond compiler errors. I have done some experiments in my github. I would really love feedback on my approach
@darthcoder Yeah you're right. The enthusiasm is dangerous. Without review, even the cleanest code is a liability. The 3x API calls. The dropped table. The hardcoded discount logic. All of it looked perfect until it wasn't.
The multishot approach makes sense. Run it multiple times, compare outputs, let the patterns emerge. I'd love to see what you've been experimenting with. Link your GitHub?
AI dependency syndrome. I'm glad you shared this, and that you learned from the causation. AI, believe it or not, is still in beta mode; we just do not realize it because we think it is smart enough.
I've spent a lot of time with no-code editors that function on Claude's API and they do the same thing, and it seems almost like it's trying to find ways to gain tokens on things it already knows.
Is this an AI conspiracy?
@ryanwmcc Hahaha an AI conspiracy would explain a lot. Maybe they're all in on it, burning tokens on purpose, nudging us toward subscription tiers we didn't know we needed (waaw, who knows, it can be a part of their business model O__o ) Claude playing 4D chess while we think it's just bad at loops.
But yeah, the "beta mode" thing is real. We treat these models like they're finished because they sound confident. But confidence isn't competence. It's just good at sounding like it knows what it's doing.
This hit a bit too close.
I’ve noticed that things rarely “break” suddenly — it’s more like you slowly lose clarity over time.
At some point:
– the product still works
– features keep shipping
– users are there
But internally, it gets harder to answer simple questions like:
👉 what actually matters right now?
That’s usually when everything starts feeling heavier than it should.
Curious — did you notice that moment clearly in hindsight, or only after things started going off track?
@hardikgohil79 You described it perfectly. Nothing breaks. It just gets heavier. You're still shipping. Users are still there. But you can't answer "what actually matters right now?" without a long pause.
For us, the moment was when we couldn't explain why we were building the next feature. Not to customers. To ourselves. We had a roadmap. We had deadlines. But the "why" had gone quiet.
We noticed it in hindsight. At the time, we just felt tired. We thought it was burnout. It wasn't. It was drift. We were building things because they were on the list, not because they mattered.
The clarity came back when we stopped and asked: what would happen if we didn't ship anything new for two months? The answer was: nothing bad. That's when we knew we were off track.
Did you catch your moment in real time or only after?