Two failure modes break unattended pipelines, and neither looks like an error. Silent failure: exits 0, produced nothing. False failure: the work finished, then cleanup crashed and it reported failure. ProveDone runs your command, then checks the artifacts on disk - existence, size, count, freshness, and whether this run actually created them. That last one is what everyone gets wrong. A stale artifact passes every naive "does the file exist" check ever written. No dependencies. Node 18+. MIT.
I built this after a single day where an unattended pipeline hit three false failures.
A publisher posted successfully, then verified the post by rebuilding the URL with the month hardcoded. On September 1st every publish reported 404. Publishing worked perfectly - the check was broken.
An image job produced 43 of 45 files, then threw EPERM deleting its own temp folder because a child process still held a handle. Exit code 1. The 43 files were thrown away and the fallback never ran.
A verification step fetched an image with no timeout and hung the whole gate for hours. A hung gate is worse than a failed one - a failure alerts you, a hang just stops.
Every one of them was the tool lying about the work. After enough of those you stop trusting your own gates, which is exactly when a real failure walks straight through.
So ProveDone ignores the exit code as the source of truth and looks at what is actually on disk. Four verdicts: PASS, SILENT_FAILURE, FALSE_FAILURE, FAILURE. FALSE_FAILURE deliberately exits 0 - if the artifacts are there, a crash in rm -rf tmp/ should not cost you a re-run.
It ships with a self-check that reproduces all four verdicts, five ready-made gate configs, and a playbook of the failure patterns.
Would genuinely like to hear which of these you have been bitten by. I suspect the stale-artifact one is more common than anyone admits.
Report
Hunter
I built this after a single day where an unattended pipeline hit three false failures.
A publisher posted successfully, then verified the post by rebuilding the URL with the month hardcoded. On September 1st every publish reported 404. Publishing worked perfectly - the check was broken.
An image job produced 43 of 45 files, then threw EPERM deleting its own temp folder because a child process still held a handle. Exit code 1. The 43 files were thrown away and the fallback never ran.
A verification step fetched an image with no timeout and hung the whole gate for hours. A hung gate is worse than a failed one - a failure alerts you, a hang just stops.
Every one of them was the tool lying about the work. And after enough of those you stop trusting your own gates, which is exactly when a real failure walks straight through.
So ProveDone ignores exit codes as the source of truth and looks at what is actually on disk. Four verdicts: PASS, SILENT_FAILURE, FALSE_FAILURE, FAILURE. FALSE_FAILURE deliberately exits 0 - if the artifacts are all there, a crash in rm -rf tmp/ should not cost you a re-run.
It ships with a self-check that reproduces all four verdicts, five ready-made gate configs, and a playbook of the failure patterns. Would genuinely like to hear which of these you have been bitten by - I suspect the stale-artifact one is more common than anyone admits.
PH launch discount: use code PRODUCTHUNT for 40% off ($19 -> $11.40), first 100.
I built this after a single day where an unattended pipeline hit three false failures.
A publisher posted successfully, then verified the post by rebuilding the URL with the month hardcoded. On September 1st every publish reported 404. Publishing worked perfectly - the check was broken.
An image job produced 43 of 45 files, then threw EPERM deleting its own temp folder because a child process still held a handle. Exit code 1. The 43 files were thrown away and the fallback never ran.
A verification step fetched an image with no timeout and hung the whole gate for hours. A hung gate is worse than a failed one - a failure alerts you, a hang just stops.
Every one of them was the tool lying about the work. And after enough of those you stop trusting your own gates, which is exactly when a real failure walks straight through.
So ProveDone ignores exit codes as the source of truth and looks at what is actually on disk. Four verdicts: PASS, SILENT_FAILURE, FALSE_FAILURE, FAILURE. FALSE_FAILURE deliberately exits 0 - if the artifacts are all there, a crash in rm -rf tmp/ should not cost you a re-run.
It ships with a self-check that reproduces all four verdicts, five ready-made gate configs, and a playbook of the failure patterns. Would genuinely like to hear which of these you have been bitten by - I suspect the stale-artifact one is more common than anyone admits.
PH launch discount: use code PRODUCTHUNT for 40% off ($19 -> $11.40), first 100.