LangWatch
AI Agent Testing, Observing & Evaluation
Start new thread

Claude Code usage tracking by LangWatch - See what your Claude Code sessions actually cost

by•
Track Claude Code usage: cost, cache, session replay. Run `npx langwatch claude` once. Every session gets cost with cache reads/writes as separate token classes, every bash and MCP call as a span, theoretical vs billed for your Max plan, and a full terminal replay in the UI. Works for Codex too.

Add a comment

Replies

Best
Maker
📌

Hey Product Hunt, Manouk here, founder of LangWatch.

We built this because we run a fleet of coding agents ourselves and had no idea what they actually cost: subscriptions hide the number, and local trackers only see one machine.

You know what Claude Code costs you per seat. What you probably can't answer is which sessions burned the budget, which model did the work, or whether your cache was earning its keep. Today's token prices are heavily subsidised, and the teams who find out what they actually consume after prices move are the ones who get surprised.

This week we've added a new launch in LangWatch, which our dev-team is using day-in, day-out. You can now track your full Claude Code usage on LangWatch: every session, tokens spent throughout the month, cache hit analysis, bash commands, skills and MCP tool calls, with a full reproduction of your terminal right in the UI.

Getting started is one command:

npx langwatch claude 

From there, every session lands in LangWatch automatically:

Where the tokens go: cost per session and per model, with cache reads and writes accounted separately.

Theoretical vs billed: on a Max plan, see what your usage would have cost at API prices.

Every tool call: bash commands, file edits, skills, and MCP calls, each as a span with duration.

Terminal replay: step through the whole session in the UI, as it happened.

Works the same for Codex, Gemini CLI, and opencode.

Free for individual use.



I'll be here all day, happy to answer anything, and genuinely curious what you'd want to see next in it.

Congrats on the launch  !

Maker

 thanks!!

  How are you keeping secrets out of the terminal replay when commands and environment output can expose tokens before the session reaches the dashboard?

Maker

hey , LangWatch has automatic PII and secret redaction of all the data, so api keys, tokens, private keys, db urls, all get redacted. Plus you can add custom patterns for you and your team to scrub. It's redacted before getting stored on LangWatch side.

Theoretical vs billed is the number that decides whether you stay on Max or move to the API, so I'd put that one on the front of the dashboard rather than three clicks in. The part that will bite you is the price table itself. If you reprice historical sessions when a model's cache read rate changes, last month's report quietly changes too, and that's a number people build budgets on. Stamp each session with the rates that were in force when it ran.

Maker

 let us know what your outcome is! ;) thanks for sharing.

 Will do. I'm putting it on a week of real work next, and the number I actually want out of it is whether the cache is earning anything, because right now I have no feel for that at all.

Maker

 oh that is in the front actually, on the home of the app! I just skipped on the demo video. But it works as you expected, rates are stamped and immutable as the sessions arrive, so you can compare month over month

indeed companies and even the engineers themselves are "feeling" the pricing dynamics under them, but without really having any visibility as it's all hidden under a plan and evolving model behavior, which is dangerous liability to have when token sponsoring ends

   The stamped-immutable-rate answer covers the common case well. Curious about a session that straddles a price change though, one that starts before a rate update and keeps running after it takes effect. Does the whole session get stamped with whichever rate was active at the start, or does it end up split across two rate periods internally?

     this is the one I'd want nailed before I trusted a month-over-month chart. The stamped-immutable-rate answer is right for the common case, but a long session that starts before a rate change and keeps running after it — single rate stamped at session start, or split internally across two rate periods? On one session that's rounding. On a fleet running overnight it isn't.

Ryan's "cost per outcome, not cost per session" point below is the other half of this for me, coming at it from a different angle. I route deliberately by model — cheaper models to gather context, expensive ones to judge. What I've never been able to answer is whether that routing actually pays for itself, because the win shows up as output quality and the cost shows up in tokens, and I've never had both in one view.

So the thing I'd want: can sessions be tagged or grouped, so I can compare two routing strategies against each other rather than comparing this month to last month? Per-model cost gets me halfway there. That comparison is the report I'd actually open every week.

   My instinct is the rate should attach to each call as it lands, not to the session wrapper. Then a straddling session is just a sum of calls at two rates and it stops being a special case at all. Whichever way it actually works, it needs writing down, because people will assume the version that flatters their number.

 Good, that's the right call. The liability point is the real one, most teams are running on a price they never negotiated and can't see, so the first honest number they get is going to sting. At least with the history stamped the sting is measurable instead of a vibe.

This could become essential for teams trying to optimize AI engineering budgets. Congrats and team!
Maker

 that's indeed the goal, so additionally to the current usage insights for the users, we do provide Admin dashboards as well to view the spend accross teams, projects and so on.

Congrats! How much overhead does the tracking introduce during long-running Claude Code sessions?

Maker

hey  Good question, we don't sit in the request path, so nothing is added to the model call latency. We read the session data Claude Code already writes locally and ship it in the background in batches, so the overhead is a small background process rather than anything you feel per turn. It stays flat over long sessions since we tail incrementally instead of re-reading the whole transcript.

Congrats on the launch!
Token tracking is becoming crutial with everyday usage of Claude... maybe eventually it will turn out that junior developers are less expensive after all :)

P.S:
I have built an interactive livedemo for you, feel free to check it out

Maker

 Thanks quick, thanks for sharing that, we'll have a look at it at the same time, please feel free to start using and let us know how it helped you!

Good to note that individual developers can use it for free before rolling it out to larger teams. The pricing that I see on the top nav is for LangWatch or for this specific tool?

Maker

 it's for the whole platform, so you get LLMOps + Gateway + Governance on a single place ;)

💡 Bright idea

This feels like the equivalent of DevTools, but for AI-assisted development. Congratulations on the launch

Maker

   💯

Manouk's "we run a fleet of coding agents ourselves and had no idea what they actually cost" is the honest version of this. Same here — our answer was "we'll find out when the subsidies stop."

One push: cost per session can mislead. The number that changes how you route work is cost per outcome — per merged PR, per green CI run, per fix that actually held. A run that spends 3x on the expensive model and lands it first try beats a cheap model looping four times — but a per-session view shows the opposite, and quietly trains you to downgrade the model that was earning its price.

You're already logging tool calls as spansm that: span → the commit or test run itproduced. On the roadmap, or are you deliberately staying at the session boundary?

+1 to the immutable price-stamp point above, with a wrinkle: it has to cut both ways. Rates must never change historical reports by accident — but "what the new prices" is exactly the question the subsidy argument makes people ask. That's a deliberate re-price, not a mutation.

 Cost per outcome is the harder number and the right one. The catch is the outcome gets defined outside the tool, a merged PR or a green CI run, so it has to accept a signal from your pipeline rather than try to infer one from spans. On the re-price wrinkle, a deliberate what-if is fine as long as it lives in its own view and never overwrites what last month's report already said.

 That's the answer I was hoping for — inferring "this span became that merge" from the spans alone is where these usually go wrong.

One thing worth designing early if you do take a signal in: the outcome shows up long after the spans close, sometimes hours, so it's a late join rather than a field written at emit time. Which makes the correlation key the real product decision — something that survives out of the agent, into the branch or the CI job, and comes back attached to the result. The ingest endpoint is the easy half.

And having the outcome defined outside the tool feels like the right place for it. You're the join, not the judge. Congrats on the launch.

 The key that survives is the branch name. It's the one thing the agent, the branch and the CI job all already know without anyone writing glue for it, and commit SHA looks better until someone rebases. Late join is the part I'd underestimate though, because it turns span retention into a product decision rather than an infra one. If you only keep raw spans 30 days you've quietly capped how slow an outcome is allowed to be.

Congrats on launch! Can it dig through my year worrh of session data to analyze my usage patterns retroactively, or does it start tracking only once installed?
Maker

 it's after installed, moving forward.

Looking back to previous session data could allow you to estimate tokens and cost and capture the inputs and outputs for sure, but LangWatch hooks up on the actual api return and telemetry of claude code (or codex etc) so we get the actual token spent and a lot other metadata information directly from anthropic (or other providers) which is perfectly accurate

we are thinking of making a utility to help with retroactive imports though! Let me know how valuable would that be for you vs just going forward

thanks! It’s more of a nice-to-have feature.
Cache hit analysis is something I've wanted to measure for a long time. Nice work!
Maker

 whoop, let us know once you've set it up, there is so much more to explore! thanks for ssharing :)

1234
Next