All activity
NorthernDevstarted a discussion
Built a middleware to catch LLM hallucinations in RAG apps
I've been working on RAG applications lately and the biggest issue I keep running into is trust. The models sound confident, but the facts are often slightly off, and manually checking logs is not sustainable. So I built AgentAudit to solve this for my own projects. It is a middleware API built with Node.js and TypeScript that sits between your LLM and the user. It uses PostgreSQL and pgvector...
Stop AI hallucinations. AgentAudit is a middleware API that acts as a semantic firewall for your agents. It intercepts LLM responses and verifies them against the source context in real-time. Catch silent failures before they reach your users. Built with TypeScript & pgvector.
AgentAuditThe "Lie Detector" API for RAG & AI Agents
NorthernDevleft a comment
I built AgentAudit because I kept running into the same frustrating issue with my RAG agents: silent failures. You know the scenario. The retrieval works perfectly, the context chunks are correct, but the LLM still confidently hallucinates a specific detail that isn't in the source text. You often don't find out until a user reports it. I tried fixing this with extensive prompt engineering, but...
AgentAuditThe "Lie Detector" API for RAG & AI Agents