If your product can fail on its own, who pays for the retry?
Anything generative has a failure rate you can't fully engineer away. Mine is low but it isn't zero, and when it misses, the user has already spent one of a small number of credits.
I've gone back and forth on this and landed somewhere I'm still not sure about.
What I considered:
Free retries. Cleanest for the user and the one I wanted. The problem is a retry costs me exactly what the original cost, and "free" has no natural stopping point — someone who doesn't like the output can re-roll indefinitely, and I can't tell dissatisfaction apart from someone farming variations.
One free retry per generation. Fairer, but it turns every borderline result into a judgement about whether it "counts" as a failure, and I'd be the one judging.
Charge for it, and price credits assuming a share of them get spent on retries. This is what I do now. It's honest about the compute, but it means a bad result costs the user something on the day it happens to them, which is the worst possible day for that.
Refund on request, handled in support. No policy, no automation. Doesn't scale, but might be correct at my size.
Where I'm stuck is that I can't measure the thing I'd need to decide. I don't know what share of retries are genuine misses versus taste. I could ask on the retry — "what was wrong with it?" — but I suspect that mostly annoys the people I most want to keep.
For anyone running something generative and metered: how do you handle a failed generation? Do you charge for the retry, and if you don't, what stops the re-roll loop?
Replies