We built a model to generate 1,000 questions that people actually ask. Not random prompts. We scraped 50,000 real user queries from search logs, forum threads, and support tickets across 12 industries. We clustered them by intent and generated 1,000 representative questions.
We asked those same 1,000 questions to 5 AI models: ChatGPT (GPT-4), Gemini (Ultra), Perplexity (Pro), Claude (4.5 Sonnet), and Llama (3). We ran the experiment daily for 30 days. We tracked every citation at the source level.
The goal: measure citation overlap. How often do these models cite the same source for the same question?
Right now we have scenarios covering things like giving hard feedback, managing up, and pushing back on scope creep, and more. But I'm building out the next set and I'd rather build what people actually need than guess.
So: what's the conversation you keep putting off?
What's the one you replayed in your head after it went sideways?
Six months ago, we ran an experiment with our own data.
At Rankfender, we tracked 5 of our own competitors across 8 AI systems. We log their share of voice, citation velocity, content gaps, platform variance. Months of raw numbers sitting in a dashboard.
I pulled 6 months of data and fed it into Claude. One question: "Based on this, who is most likely to overtake us in the next 6 months? Show your work. Use the data. Don't summarize. Give me the numbers."
Control group: a two-hour roadmap review meeting. Six people in a room (virtual). We debated features. We argued about timelines. We discussed dependencies. We left feeling productive.
Test group: We fed the same roadmap into Claude. No slides. No politics. No one trying to protect their pet project. Just the raw plan. The prompt: "Analyze this roadmap. Identify the three most likely failure points. Use first principles reasoning. Assume we will follow your recommendations without ego. If you need more data, ask for it."
Just think for a sec. You've told different chat agents your role, your tech stack, your client preferences, your project constraints - hundreds of times across hundreds of conversations.
But where does all that live?
Scattered across chat histories. Fragmented across different platforms. Sometimes contradictory, & mostly out of date.