When your AI product says a job is Completed, what has it actually promised?
Nico’s thread this week (green tests, 5.5 hours down) reminded me of a smaller version of the same problem on the B2B AI product I manage.
The scene: a data preparation job finishes and the row turns green with “Completed”. The customer reads that as “checked and safe to use”. It means “ran without errors”. Weeks later someone asks why a model was trained on a version nobody signed off on, and the status page says everything was fine.
Why it’s hard: one status is doing three jobs. It ran. Someone or some check confirmed the output is good enough for this task. Something downstream is using it. Different people decide those at different times, so they diverge: complete and never used, or in use before anyone checked.
What we’re leaning toward on our side (one team’s guess, not a rule): three separate statuses with their own owner, and “checked” cannot be set by the job that produced the output.
-------------------
Two questions:
In your product, what does Completed literally mean, and have users ever read more into it?
If you track “checked” or “approved” separately, who is allowed to set it: the system, the user, or a reviewer?
Replies
we hit the same thing with tool approvals.. a baseline recorded on first sight and one a person approved looked identical, so the warning said you approved it when nobody had. now it says who set it, and approve refuses when it can see an agent running it. do you store who set checked, or only that it is set?
In my song app "ready" literally means the audio file exists, and I found two places where people could read more into it. The lyrics on screen: the music model returns the lyrics it planned to sing, not what it sang, so a separate step listens to the finished track and corrects them. That step is best effort, if it fails the song is still ready and shows the planned text. So ready does not mean checked, exactly your split. The other one is your "in use" status: the home screen widget says "Ready, tap to listen" until the song has actually been played, then switches to the style line. Ready and heard are separate facts there, set by different events.
Same split here. When my agent posts something, a successful request never counts as done. It reloads the page and only marks the item posted once it can see the text there. Today that caught two replies whose parent comments had disappeared. The page check runs after the post as its own step.