Can your AI-generated test actually fail?
I built Code Factory after losing time to AI-generated tests that passed but failed in real use. The front door is now deliberately simple: run `factory first-proof --root .` and see a safe demonstration of whether a test can actually reject a known failure. It stays local and does not change your project.
For a solo developer or vibe coder, that is the whole starting value. For a team, `factory wrap` records the exact file delta from an admitted agent run, runs the validators the team declared, and leaves a reviewable receipt without retaining prompts or raw model output. Platform and assurance teams can evaluate the deeper policy and evidence controls in a pilot, but this beta does not claim an SLA, certification, customer references, or procurement readiness.
Version 0.44.3 also compares declared and observed product journeys, preserves the failed step in a bounded evidence capsule, checks state and cleanup, and audits the repairing agent separately. Human approval, apply, merge, release, credentials, and deployment remain outside Code Factory's authority.
Try it: https://pypi.org/project/factory...
Code Factory 0.44.2: trace verified intent back to source
A small but practical review update: Graph Ops can now take a verified Forge intent trace back to the exact hash-bound source line.
If the binding is missing, malformed, or mismatched, the source action stays unavailable instead of guessing. It is read-only: no repair, approval, deployment, or credential access.
This helps when an AI-assisted change looks finished but you still need to answer: what approved intent does this result actually come from?
Release and install: https://github.com/zrk222/code-f...
Code Factory 0.44.1: current UI, First Proof, and a transparent 60-day use case
I built Code Factory for myself. Now I m sharing it with the world.
AI made code generation dramatically faster, but generating code was never the whole job. I still needed to understand what changed, catch incomplete work, preserve evidence, and know what was actually safe to accept.
Code Factory 0.44.1 updates the public product surfaces and brings the current workflow into focus. First Proof runs a sealed positive control and an intentionally hollow negative control. If the broken path still passes, Code Factory reports HOLLOW_E2E_TEST instead of calling the run proven.
Expected outcomes:
Graph Ops now compares proof-backed repairs before code changes
I built Code Factory for my own work after wasting too many hours on AI-generated code whose tests passed but failed in real use. I wanted to see what was actually proven, what was only claimed, and where a run diverged. I am sharing it free so other builders do not have to repeat that frustration.
Version 0.32 adds ProofSearch, a read-only Counterfactual Arena inside Graph Ops.
It rejects repair candidates with failed proofs, surviving mutants, changed evidence, or edits outside the approved path slice. It then compares eligible candidates deterministically by risk, changed lines, proof time, and measured usage. Every winner, loser, and rejection is explained. Apply verified repair remains locked until a human reviews the result.
For vibe coding, it answers: which AI fix should I trust? For professional teams, it creates a reproducible, hash-bound repair decision with explicit authority boundaries.
v0.31: Make vibe-coded software prove it can fail
Code Factory v0.31 is for the moment after an AI coding tool says done.
For vibe coding, it helps turn a prompt or PRD into a contained local MVP, then checks whether the evidence can actually reject a bad result. The new `factory e2e verify` gate requires a passing path and a deliberately broken path. If the broken path still passes, it reports `HOLLOW_E2E_TEST` a test that ran without proving anything useful.
For teams, it adds a practical operating layer around AI-assisted changes:
Plan-to-Proof compares the real diff with a human-approved plan and records scope, test, and evidence gaps as Proof Debt.
v0.28: Proof Review and Verified Repair Sandbox
Code Factory v0.28.0 is out.
It adds three supervised, local controls for the hand-off between a code change and independent review:
Proof Review turns the current diff into JSON, Markdown, and Mermaid evidence.
Verified Repair Sandbox binds one native Change List and checks candidate patch scope and measured bytes.
