“It Worked Once” Is Not The Same As “It Is Repeatable”
If you’ve been following along and want to support us, we’d really appreciate a GitHub star:
A repo reaching a working state once is useful.
But the real value comes when that working state can be repeated by the next person, on the next machine, or in the next automation run.
That is where many repos drift. Someone fixes setup locally, but the path is not captured. A dependency changes. An env value is missed. CI and local dev slowly stop telling the same story.
Ota’s `ota up` is focused on preparing the repo based on the contract, so setup is not just a set of instructions someone followed once. It becomes something the repo can describe and repeat.
The line we keep coming back to is:
"Make the first working run repeatable."
What usually causes setup drift in the projects you work on?
Replies
The biggest source of setup drift I've seen is undocumented environment variables. Everything looks fine until a new teammate spends hours figuring out which secret is missing.
Ota
@alan_gregory I agree. There's also no guarantee the documentation hasn't already drifted. That's exactly the kind of setup drift we're trying to catch with Ota. The problem usually isn't just that a variable is missing. It's also not knowing where it comes from, which environment needs it, or whether there's a safe default. With Ota, those requirements can be declared and checked before someone gets stuck. We wrote about that in more detail here: https://ota.run/blog/how-ota-prevents-environment-variable-drift-in-repo-setup-6f2d
Usually it’s the hidden contract, not the command: env vars, seed data, external services, and the thing someone fixed locally but never encoded. The useful artifact is a setup receipt that can fail when drift comes back.
Do you treat `ota up` failures as contract failures, or more like repair suggestions?
Ota
@blah_mad Yes, exactly. Treat `ota up` failures as contract signals first, with repair guidance second. If `ota up` can't make the selected path ready from declared truth, that usually means one of three things: a declared prerequisite is missing, the repo still depends on something never encoded, or the contract has drifted from reality. That's also why receipts matter. The useful artifact is not just "setup failed," but evidence of whether the failure came from declared requirements, hidden dependency, or drift. We wrote a short note on that here: https://ota.run/blog/why-ota-treats-setup-failures-as-contract-signals-2p7n)
And the receipt surface is here:
https://ota.run/docs/reference/execution-receipt
How do you think about the line between "hidden dependency" and "contract drift" in practice?
I draw the line by whether the assumption ever had a home. Hidden dependency is an undeclared thing the repo secretly needs: env var, service, seed data, local order. Contract drift is when the contract had that truth, but reality moved and the check no longer catches it.
Do Ota receipts version the contract too, so a fix can point to the assumption that changed?
Ota
@blah_mad Yes, partly. Ota receipts does carry contract identity, so a run is tied back to the contract surface Ota believed it was executing against, not just "setup failed." The archived receipt lane preserves that too: https://ota.run/docs/reference/execution-receipt The honest nuance is that this is compact contract identity, not full assumption-level versioning yet. So a receipt can anchor the run to the contract in play, but for the exact assumption that changed you would still pair it with contract diff and Git history. That is also the direction we are pushing: stronger receipt evidence, without pretending the receipt alone is already the whole change ledger. Curious what you'd want the ideal artifact to look like there: a receipt that carries the exact contract snapshot, or a receipt-plus-diff model where execution evidence stays separate from contract-change evidence?
@faithada I’d keep the receipt-plus-diff split.
The receipt should say what actually executed: contract id/hash, repo ref, machine/runtime, step, outcome, and artifact links. The diff should say why the contract changed: old/new assumptions, author, reason, and review. The receipt can point to the immutable snapshot without carrying the whole thing.
Bad but common case is when the behavior is split between a number of custom scripts and only the initial developer knows in what order and under what conditions they should be executed to run the project. Personally in the projects I work on I try to maintain single entry point in the form of Makefile, and secrets should be provided via a single .env file that you can share with a colleague if needed. I also maintain .env.example to make clear what is required for the project.
Ota
@dani_polani A single entry point and a maintained `.env.example` already put you ahead of a lot of repos. Where Ota goes further is making the order, prerequisites, env truth, services, and verification path explicit in the repo contract itself, instead of leaving that knowledge partly in `Makefile`, partly in scripts, and partly in maintainer memory. Our framing is: - `Makefile` is a strong front door - `.env.example` is a good baseline - the next step is making the repo declare what is required, what is safe, what services must be ready, and what proves setup actually worked That's the gap Ota is trying to close. Curious in your experience: does drift happen more in task order, or in hidden env/service assumptions?