Forums
What are you building, and what does your stack look like?
I am a Computer Science student doing research into how solopreneurs and small startups create new apps and what their stack looks like. Particularly, I'm interested in how you handle things like authentication, billing, and permissions/authorization in your apps.
Let me know what you're working on below and how you're going about it -- I'd love to connect for some quick calls to learn about your product and talk about your process in building it!
How are you measuring your AI drift?
It's a proven fact that none of the AI systems breaks overnight; They decay. They fade, shift, and degrade quietly.
Stanford found GPT-4 accuracy on basic reasoning tasks dropped 97.6% -> 2.4% between March and June:
https://arxiv.org/abs/2307.09009
variA/Bly has evaluated across 10+ workflows, and the same pattern appears:
Accuracy drifts (almost 15 40%), prompts regress, RAG relevance drops, and costs fluctuate (20 50%).
The real truth:
The fact is that AI systems are inherently indeterministic, and hence the drift is natural.
The real business risk is that most of the business owners aren t measuring it.
Recently, we launched a 30-day "AI Drift & Accuracy Pilot" to help teams see how their workflows change week to week.
If you want your drift map, happy to share.
