Have high AI inference costs ever forced you to trim or rework a feature?
by•
Seeing big tech companies secure whole power plants for AI compute is a reminder that running these large models takes serious resources.
It made me rethink the common assumption that model costs will infinitely drop over time.
I started adding a column in my metrics sheet to track exact AI costs per user interaction. Turns out one fancy background feature was burning through tokens without adding real value to the user experience.
Are you actively monitoring token costs per feature in your app or relying on price drops to keep your margins safe?
12 views
Replies
this is something I’d want to track from the beginning. My costs could look manageable overall while one particular workflow is responsible for most of the spending.