Are you using AI to stress-test your idea before writing a single line of code?

by

I've noticed a pattern in almost every "how I built this" thread here lately: before touching the keyboard, people are opening a chat window with an LLM and asking it to poke holes in their idea.

Makes sense. Vibe coding has made building fast. You can go from idea to working prototype in an afternoon. But building fast doesn't mean you're building the right thing faster. If anything, it raises the cost of a wrong guess, because now you can ship the wrong product in a day instead of a month.

So I've started using AI earlier in the process to argue with me. I'll describe an idea and ask it to play the most skeptical potential customer it can, or to list every reason the idea might already be a solved problem. Sometimes that surfaces a competitor I hadn't seen. Sometimes it forces me to actually define the problem instead of jumping straight to the feature list.

Here's the part that worries me though: AI models are notoriously agreeable. If I phrase my idea with enthusiasm, I sometimes get validation back that feels a little too easy like it's mirroring my confidence rather than actually stress-testing the logic. That's dangerous, because it can create false certainty right before you commit real time to building something nobody asked for.

So I'm curious how others here are handling this:

  • Do you use AI in your validation step, and how exactly do you prompt it to be genuinely critical rather than agreeable?

  • Has AI ever given you false confidence that led you to build the wrong thing?

  • What's your actual process for separating "AI thinks this is a good idea" from "real humans think this is a good idea"?

Not looking for tool recommendations here, just more interested in the actual habits and prompts people use, and where you've drawn the line between AI-assisted thinking and just talking yourself into something.

46 views

Add a comment

Replies

Best

I use AI to generate objections, not verdicts. I ask it to identify assumptions, propose cheaper explanations, and describe what evidence would disprove the idea. Then I test the riskiest assumption with real users. The model improves the questions, but user behavior decides whether to build.

  Exactly. I think the distinction between generating objections and asking for a verdict is crucial. AI is much more useful when it helps us identify what to test rather than telling us what to believe. The real signal still comes from how people actually behave.

 The "objections not verdicts" framing is really clean. We do something similar, ask for the riskiest assumption buried in the idea, not a list of problems. Usually surfaces one thing that, if wrong, kills the whole concept. That's the thing worth testing first.

I never stop at one conversation. I rewrite the same idea from different customer perspectives and compare the answers. Have ypu noticed different roles exposing completely different wealness in the same product concept.

  Yes, absolutely. I’ve found that changing the perspective can expose weaknesses that a generic validation prompt completely misses. A buyer, end user, and decision-maker can all see the same product very differently. That’s often where the most useful questions come from.

 Yes, totally. The buyer vs. the end user perspective alone can flip whether something feels like a painkiller or a vitamin. We've had cases where the person paying loved the idea and the person using it daily had zero interest. Rewriting from different roles caught that way earlier than we would have otherwise.

I found that challenging every feature with one simple question improves decisions. Would someone pay for this today. If the answer stays unclear I keep researching before writing any code.

 "Would someone pay for this today" is genuinely underrated as a filter. The word "today" does a lot of the work, it cuts out the "yes in theory" answers and forces you to think about urgency. If the honest answer is "maybe in 6 months when they have budget," that's basically a no for now.

Totally agree on the false confidence risk. One thing that shifted things for us: instead of asking "what's wrong with this?" (which still gets polite answers), we ask the AI to write a one-paragraph teardown from the POV of a skeptical investor who just lost money on a similar bet. The framing change makes the output noticeably less agreeable.

Still not a replacement for talking to real users, but it surfaces different objections than the generic "here are some challenges you might face" response.