If your product charges per request, how do you handle retries?

by•

When a program retries on its own, a lost response looks exactly like a failed call. For free APIs that is harmless. For anything that charges per call, a retry after a timeout can mean paying twice for one result.

Building paid tools for AI agents, I ended up relying on the payment itself being single-use, plus an idempotency key that returns the stored answer for a while. The part I'm still unsure about is how long that window should be. Too short and a slow recovery gets no answer; too long and you keep a lot of state around.

If your product charges per request or per action, how do you handle retries? Do you key on the payment, the request, or something else, and how long do you keep the result?

46 views

Add a comment

Replies

Best

I'd base the window on how long your slowest client keeps retrying, then add some slack. And store a pointer to the result, not the whole response, so a longer window doesn't get expensive.