Do Vibe coders actually test things?

I'm usually asking Claude to write unit tests and integration tests after it wrote some code under my supervision - therefore trying to always stay covered :)

For me, the real question isn't whether vibe coders test things, it's how they test them. I always run through real scenarios because code that looks right isn't always code that behaves right.

I still test the important flows myself especially when APIs or user data are involved. For small internal tools I move faster but I don't fully trust agent output w/o checking it. Do you use a separate agent just for breaking/testing the app?

For testing, creating good evals is critical! Both offline & online eval dashboards. Online evals run with every user so you can see performance in real time. Offline evals are created to pressure test before launch, so that users don't break things to begin with. This Lenny's podcast episode on evals was super useful for me: youtube.com/watch?v=BsWxPI9UM4c

Honest answer: I test the happy path and let users stress test everything else. The agent writes it, I click through the main flow, ship it, and wait for something to break.

It's not great practice but it's the reality when you're moving fast solo. The upside is bugs surface faster when real people are using it. The downside is sometimes real people are using it when it breaks.

Haven't gone as far as delegating testing to another agent yet but I've seen people doing that and it actually makes sense. If the agent wrote it, another agent probably knows where the weak spots are better than I do.

Endpoints? Barely. That's usually where things catch up with me first.

Honestly, I think the answer depends on what you mean by "test." I run my vibe-coded stuff through the happy path and maybe 2-3 edge cases manually, then ship it. For internal tools and prototypes, that's been enough. For anything touching user data or external APIs, I'll ask Claude to write unit tests after the code is done. Not perfect but it catches the obvious stuff.

The real gap I've noticed is integration testing. The AI writes clean isolated functions but the moment two pieces need to talk to each other, things break in ways that unit tests don't catch. That's where I still end up debugging manually, which kinda defeats the speed advantage. Anyone found a good workflow for testing the "glue" between components?

Interesting question.

We see a similar pattern in AI filmmaking.

The bottleneck used to be creation.

Now the bottleneck is validation.

AI can generate a script, a character, a scene, even an entire short drama.

But determining whether the story will resonate with real audiences is still surprisingly difficult to automate.

Makes me wonder if the next generation of AI tools won't be better generators, but better evaluators.

Looking forward to seeing how you're approaching this problem in the API and developer tooling world.

I think it depends a lot on the maturity of the vibe coder.

For me, the foundation is a spec-driven approach. Before any coding starts, I need clear user scenarios documented in Markdown files. The spec becomes the source of truth.

After that, I either go through those scenarios manually or ask agents to execute and verify them. So yes, testing is still happening — often more systematically than before.

The bigger issue, in my opinion, is that many projects don’t have clearly defined scenarios at all. If the expected behavior isn’t documented, neither humans nor AI can reliably test it. That’s where most testing problems actually begin.

most dont, and the ones who do usually only check the happy path. the agent writes code that runs, you watch it run, and you quietly treat "runs" as "correct". theyre not the same thing.

what works for me is deciding the edge cases before i prompt, then checking the output against that list. if you let the same agent write the code and decide what to test, youre asking it to grade its own homework. the spec has to come from you - or from a second agent thats given the requirements but never sees the code.

From my experience, letting a few real users join the Beta is the best test out there. They will break everything that is breakable... and a few things you thought were mathematically impossible to break! The less "tech-savvy" your beta users are, the faster you will see exactly where they get stuck in your interface. 

As a solo full-stack engineer, I try to balance automated guardrails with real-world chaos. I’m currently building Trackly (a B2B compliance SaaS) and honestly, my favorite "vibe test" was generating a unique QR code for my dog Shaman’s collar to track his vaccinations schedule. If a furry office supervisor can't break the flow, there's hope for the enterprise clients! 🐶