Launching today
Theia
Catches your AI's hallucinations and flattery, live
3 followers
Catches your AI's hallucinations and flattery, live
3 followers
Theia is a Chrome extension that was designed to check LLMs like ChatGPT, Gemini, and Claude for hallucinations and sycophancy. Within a few seconds, users can analyze the replies of any of these AI for misleading facts or a flattering tone, with each analysis backed by numerous sources.
Hi all,
I built this because I got tired of how AI was constantly agreeing with me. I'd push back on an LLM about something true, and it would just change its mind in an instant. That's useful for a yes-man, not for something you're trying to get facts from.
With this extension, users can analyze each message of their AI with just the click of a button. Theia uses sourced fact-checks: pull a claim and Theia checks it against numerous sources while also showing you the actual links to where it derived its information instead of just asserting an answer.
On the side of the user's chat interface, users will also see a sycophancy meter. This feature is powered by a tiny 2-layer BERT model I trained that scores each response on how much it's flattering/agreeing with you vs. being candid. It runs fully on-device.
However, this (and Theia as a whole) is still an early model that will be improved and updated, so any feedback is appreciated. Let me know: Is this a tool you'd use? What could be added/improved?
Thank you!
the source-backed breakdowns are a really smart touch, makes the results feel trustworthy instead of just another black box telling you the AI is wrong
honestly the fact that it actually pulls sources for each claim is what sold me, you know? most hallucination checkers just give a vague confidence score