Our objective was to set an A/B test and see the results in 5 minutes. We hit it. We worked hard to make setting up a benchmark easier by 1. improving model search 2. adding smart model recommendation engine 3. adding support for publicly sharing the results

Run a series of A/B tests for your LLM setup in 15 minutes. Define model, parameters and system prompt and see what's the impact on latency cost and quality. Call it a bespoke benchmark.

Narev ingests SaaS billing data and lets you export in FOCUS 1.2 format. It comes with one dashboard for everything. Self-hosted and open source so your data stays private.