When you replay or fork a run in Retrace, the steps before the fork come from the recording, but everything after runs live against the model. So two runs of the same input rarely match exactly, even when nothing actually broke.
That makes the useful question harder than it sounds: when a replay diverges, is it a real regression from your change, or just provider non-determinism? Retrace currently shows a first-divergence diff and a verdict of improved, regressed, or unchanged, but I would like to hear how others handle it. What tolerance do you use in practice, and would you rather see a strict step-by-step diff or a semantic comparison of each step?
Love how clean the replay view is, being able to scrub through each LLM call and tool invocation without losing context makes debugging agents feel way less like guesswork.
Spent a few minutes replaying a flaky agent run and being able to fork the exact trace to try a different prompt without rerunning the whole thing was honestly a nice surprise. The tool call breakdown finally makes it obvious where my agent was looping.
This might just be me coming from more of a front end and digital design background, but I don't instinctively understand what "forking" means in this context.
My brain likes to work in a linear way. When I hear fork, I imagine going off onto another path, and my first question is, "Can I merge it back later?" 😅
I wonder if other beginners have the same reaction. If someone doesn't already come from a developer or Git background, they may not immediately see the value because they don't yet have a mental model for what a fork actually is.
That may be totally fine if you're intentionally speaking to experienced developers. But if you'd like to reach newer builders or AI enthusiasts, it could be worth explaining the benefit in plain language first, then introducing "fork" as the technical term. I think that would make the product click much faster.