Hey Product Hunt! 👋 I built Weir to explore a question: can we make RAG applications cheaper without sacrificing answer quality? Weir sits in front of an existing RAG service and combines semantic caching, model routing, and per-request cost accounting. One of the biggest challenges was making semantic caching safer. Questions like "ICU visiting hours" and "general ward visiting hours" can be semantically similar while requiring different answers, so Weir uses an entity guard to reject potentially unsafe cache matches. In a synthetic, repeat-heavy 300-request replay, full Weir reduced measured cost per 1,000 requests by 94%. A separate 158-question cold pass showed a 20% saving. These are benchmark results, not production guarantees, and the repo documents the limitations and failure modes. I'd love feedback on the design, evaluation methodology, and what you'd test next. GitHub: https://github.com/yatinannam/weir

Weir

Semantic caching and model routing for RAG

Semantic caching and model routing for RAG