A few hours into our launch, Lenz is now sitting at #3 Product of the Day on Product Hunt.
For a small, bootstrapped team, this is a pretty special milestone. We re incredibly grateful to everyone who has checked out the launch, tried the product, asked thoughtful questions, and shared feedback along the way.
Paint the Cameras Dead
This feels especially important right now. We are living in a world where AI is becoming one of the first places people go to check what is true, yet your research shows that frontier models disagree on 63% of real-world fact checks.
That is such a powerful reminder that a confident AI answer is not the same thing as a verified answer.
Really love what you’re building with Lenz. The need for trustworthy verification is only going to grow as AI becomes more embedded in how we work, learn, and make decisions.
At this point, I’m just happy to have an AI that occasionally says, “Let me check that” instead of confidently making things up 😂
Lenz
@bogomep Indeed. In our experience, the models' self-reported level of confidence in their answers is mostly noise. They aren't trained to self-assess. Even worse, their incentives (to keep the user happy and engaged) push them to show high confidence when asked. On a 1-10 scale, in 76% of answers, the models report confidence of 9 or 10. And the correlation between confidence level and how much they agree with each other is not too strong either: https://lenz.io/research/llm-disagreement#confidence