How are you handling rising token costs?
Hey builders đź‘‹
Token bills are creeping up as we ship more AI features, and I'm curious how the community is dealing with it.
A few things I'd love your take on:
Model switching: Do you actually bother routing between models (e.g. a cheap/fast model for simple stuff, a premium one for hard tasks)? Or is the engineering overhead not worth the savings? Would love to hear real numbers if you've measured it.
Monitoring: How do you track consumption, and do you separate customer-facing usage from internal/employee usage? Feels like these two get lumped together and it makes it hard to know what's actually driving cost. What's your setup? homegrown dashboards, a third-party tool, tagging by request?
Please give me your lessons below ... Trying to build a mental model of what "good cost discipline" looks like right now.
Replies
We've found routing is absolutely worth the engineering effort once AI becomes a core product feature.
In Salestrics, we built a model router that evaluates each request and dynamically selects the most appropriate model based on factors like complexity, latency requirements, context size, and available budget. Most requests don't need the most capable (or expensive) model.
We also tag every AI request by organization, user, feature, and model so we can separate customer usage from internal development/testing. That makes it much easier to identify where costs are actually coming from and optimize the right areas.
The biggest lesson for us: don't optimize for the lowest token price—optimize for the lowest cost that still delivers the experience your users expect. A slightly more expensive model that completes a task correctly the first time is often cheaper than retrying or escalating multiple requests.