I build production LLM agents, and most of my week is evals, tool design and MCP integrations. Two problems kept coming back. Our agents needed reliable business data to reason over, and we needed one place to track how the agents themselves perform - eval scores, task success, latency and cost per run.
The Databox team set everything up for us. They connected our product database, Stripe and HubSpot, modeled the business metrics in the semantic layer, and created datasets for our eval results and agent traces. Our CI pipeline now pushes every eval run in through the Databox CLI, and production traces come in through the API. I didn't write any glue code.
Databox MCP is the best-designed analytics MCP I've worked with. The tools are clean, the metric definitions come with context, and our agents get consistent answers instead of guessing at raw tables. That alone cut a whole class of hallucinated numbers from our evals. A Routine compares eval pass rate, latency and cost after every model or prompt change and posts the diff to Slack, so we catch regressions before they ship. When something looks off, I ask Genie "which task types dropped in success rate since Tuesday's prompt change" and get the answer with reasoning. Eval summaries for product reviews are Artifacts Genie drafts for me.
If you run agents in production, this gives you governed data for them and visibility into them in one place.
Another step toward making analytics easier for AI agents to set up, manage, and automate!
@gregor_sulcer Thanks Gregor! That’s the direction - less clicking through setup screens, more letting agents handle the routine work.
gotta share this with my builder friends asap
@shushantlakhyani Thank you! Happy to hear the feedback
A must-have in the AI/Agentic era.
cracked team
@mcarmonas Thanks Marti!
alright, i'm all about making my terminal life easier. honestly, the thought of not having to switch tabs to check analytics feels like a dream. but 91 commands? yeah, i might just cherry-pick the ones i actually learn.
@therayess Cherry-picking is the right way to use it. Most people only need a handful of commands day to day, usually dataset ingest, metric list and metric drilldown. The skills are split by area too, so you can install just the ones you use, e.g. npx skills add databox/databox-cli --skill databox-metrics.
Getting a clear added/updated/rejected count back after every push is the part I like. It makes cron jobs much easier to trust. Does it handle schema drift when a script starts sending a new field?
@famitha Thanks Famitha! The schema is set when the dataset is created, or by the first ingest if you skip --schema, and it doesn’t change on its own after that. When records don’t match, the ingestion status shows which ones were rejected and why, so drift shows up in your cron job’s check instead of quietly ending up in your data. To add a new field on purpose, the clean path is updating the dataset setup first, then sending the new shape.
If Claude Code pushes a dataset schema change through the CLI while ingestion already has rows in the old shape, does it reject the change or migrate the existing data?
@igorblink Good question. Neither - the CLI doesn’t change a dataset’s stored schema after the first ingest. dataset update only renames the dataset. Changes like renaming columns, converting types, adding formula columns or hiding columns go through modifications, which sit on top of the stored rows. Claude Code can run dataset preview-modification first to see the result before saving anything. If new records don’t match the schema, the ingestion fails and the error names each rejected record, so nothing gets silently mixed. For a real breaking change, the clean path is a new dataset and a re-ingest.