Replay
Autonomous app testing
Start new thread

Replay QA - Replay QA tells you what is broken before your users do

by
•
Add your GitHub repo for continuous testing, or drop in a URL for a one-time check. Replay QA explores your app, records every session, finds real bugs, and gives your coding agent the root cause and fix. Try for free: qa.replay.io

Add a comment

Replies

Best

The recorded session + root cause combination is what makes this interesting. finding that something broke is useful, but giving a coding agent the exact interaction, state, and context that caused it could remove a huge amount of back-and-forth between QA and engineering.

The first-time experience also sounds almost suspiciously fast :)) Curious how Replay QA handles flaky or environment-specific behavior. can it tell the difference between a real product bug, a temporary network issue, and something that only breaks under one browser or session state?

Maker
Exactly. Finding the bug is only half the job. The useful bit is giving the person or agent fixing it enough context to avoid reconstructing the whole failure from scratch. And yes, the first run can feel slightly suspiciously good.
Maker

 One of the coding agent patterns that seems to work really well is having one AI do the work and validate itself then pass it's solution to a critic. The critic then looks at the artifacts and tears them apart. Most loops today just pass the code, some better ones will pass screenshots and logs, but passing a full deterministic recording that says "I believe this user session proves this works" seems to be best for the critic. 

Curious how Replay QA handles flaky or environment-specific behavior. can it tell the difference between a real product bug, a temporary network issue, and something that only breaks under one browser or session state?

We have been working on making this as smooth of an experience as possible. Having a deterministic recording of a flake is one of the easiest ways to tell. One of the reasons engineers hate flaky e2e test is because there is no good way to inspect the CI run to tell if it was just a flake or something more important which leads to long investigations which end with you just rerunning the job. Deterministic recordings of the browser sessions and an AI to triage them removes almost all that pain.

Hunter

 thank you for giving a try! feel free to add your review here:

Hunter

 what should they build/improve/fix from your perspective?

 From my perspective, the biggest opportunity is making the "why" behind every failure impossible to miss.

I would love to see Replay QA automatically classify whether something looks like a product bug, flaky test, network issue, or environment-specific failure, then show the exact evidence behind that conclusion: the relevant state change, request, console error, DOM difference, and what changed across reruns. A simple confidence score plus "reproduce in another browser/session" would make the result much easier to trust and act on.

The product already seems strong at recording the failure. turning that recording into a very clear, developer-ready diagnosis would probably be the thing that makes teams keep using it every day :)

Congrats on the launch, this is a sharp read on where QA broke down — AI cut build time to hours and testing never caught up. We ran into something similar: half our early bugs only showed up after the flows ran a few times, never on the first pass. The GitHub PR flow is the part I keep coming back to — root cause + fix posted right on the PR. How do you handle false positives from apps that behave non-deterministically (race conditions, flaky async UI)? Does Replay QA re-run a failure before flagging it, or does that triage still land on the dev?

Maker
That “only breaks after a few runs” category is exactly where the normal happy-path test falls apart. The goal is to make that loop much shorter: reproduce it, understand why it happened, and get something actionable back to the person fixing it.
Maker

 Replay doesn't need to re-run a failure to debug a race condition because it has the conditions which cause the race recorded.

When you have a bug which is a race condition and you submit a fix for that bug we will look at the recording in the bug and faithfully re-create the conditions which caused the race to make sure the new code actually exercises the fix before marking the bug done.

Hunter

The GitHub PR flow is the part I keep coming back to — root cause + fix posted right on the PR.

Likewise. From my perspective, this makes especially useful for engineering teams using products like , , or , who need better browser context for debugging.

The "gives your coding agent the root cause and fix" part is what stands out to me — most QA tools stop at "here's a failing session," this one seems to actually close the loop back into the fix. Did you find agents act on that context reliably, or does it still need a human to sanity-check the suggested root cause before it gets applied?

Maker
Great question. We see the analysis as context an agent can act on, not something it should blindly apply. The recording, root-cause analysis and suggested fix should make the fix much faster, but we’d still want it verified through the normal review and test workflow before anything lands. The goal is to close the loop with guardrails, not let an agent freestyle in production 🙂
Hunter

The "gives your coding agent the root cause and fix" part is what stands out to me — most QA tools stop at "here's a failing session," this one seems to actually close the loop back into the fix.

 spot on! definitely is an opinionated piece of software. S/O to makers and team!

Hunter

had the opportunity to collaborate on this launch and am blown away by how fast the first-time experience is 🤯

go to and get your first report in seconds.

what's the model you're using in the background, ?makers?

Maker

 Like most teams, we're using many models in the background as we evolve our QA harness to ensure we're using the right one for the job at hand, and to manage token burn. I'll let Brian chime in here on more technical details once it's morning on the West Coast (still 5am for him!).

Hunter

 particularly curious if you're using both frontier and open-weight models in prod 👀

Maker

   We're mainly using open weight models

This feels like a natural next step from the replay/devtools side, turning session capture into something that actively goes and finds bugs instead of waiting for a user to hit them. Since it's exploring the app on its own, is it built to dig into things like auth boundaries and cross-user data isolation too, the bugs that never throw an error and just quietly leak or drop data, or is the current focus mostly on crashes and broken UI flows? That category is brutal to catch in vibe-coded apps and it would be a big deal if an agent could reliably surface it.

Maker
You’ve nailed the distinction. Exploring flows is one thing; reliably catching auth boundaries, cross-user isolation and silent data-integrity failures needs deliberate test identities and assertions, not just an agent clicking around. That’s a hugely important direction, especially for vibe-coded apps, but I don’t want to overclaim it today. The current focus is making autonomous exploration useful for the broken flows and user-visible failures that otherwise slip through. If you’ve got examples of those quieter failures, I’d genuinely love to see them.
Hunter

 spot on! To quote 's newsletter today:

Most people shipping vibe-coded apps don't test them. They click around for ten minutes and push it live. (...) Replay QA runs that before ship: give it a URL, it explores your app, and when it finds a bug your coding agent gets a full recording of what happened, not a screenshot and a guess. The Redux maintainer is on the team. Recording over screenshots is the right foundation, and no other AI QA tool has it.

Read on:

Congratulations for launching theproduct

Maker
Thank you, Madalina. Really appreciate the support!
Hunter

thanks for the continuous support ❤️

Do we need to write test cases in our GitHub repo, or does the tool help us write the test cases for testing?
How does it work when we have feature updates

Maker

 the beauty of Replay QA is that you do not need to have even a single test case written in your Github repo. It handles this automatically. We've invested a massive amount of energy in our QA harness so teams don't have to, and can focus on more important stuff.

 if you have a free plan for OSS projects, we are happy to evaluate

Hunter

   an oss program? love the idea, oss ftw! ping

Maker

 we have a free plan period that should give you enough headroom to take Replay QA for a test drive on your OSS project. If you need more than what our free tier offers, then we can discuss. We love open-source here at Replay and many of our team members maintain and contribute to well-known projects.

Hunter

 thank you for your continuous support, Chilarai! have you experimented with the tool? would love to have your review here:

keep up the great work

Congrats on launching a neat product! Do you guys crawl read-only paths first and earn its way into write actions, or is the expectation that people hand it a scoped throwaway account and let it break things?

Maker

 When we detect the presence of auth for an app, we ask users how they'd like the Replay QA agent to proceed:

a) Provide specific username/password creds (aka, your scoped throwaway account)
b) Let the agent create its own account(s)
c) Skip auth

As you can imagine, with something as nuanced as auth and the many flavors out there, it can be tricky, but our agent handles it quite well.

Hunter

 appreciate the kind words! let's spread the word on X,

Catching bugs before users do is huge. Does Replay QA support multi-step user flow testing (like checkout paths), or is it focused mostly on page-level errors right now?
Maker

   Replay QA automatically discovers all user journeys through an app, including checkout flows for ecommerce, nuanced flows like inviting teammates to a collaborative space (think SaaS app workflows), in addition to the page-level errors.

Hunter

yes, and I'd add is especially useful for AI-built apps, indie launches, client handoffs, build-in-public demos, teams using , , , and similar tools, and developers who need better browser context for debugging.

get started here -- the first-time experience is really fast:

enjoy!

That automatic discovery of full user journeys like checkout and SaaS invites is huge. It saves so much manual test creation time. Thanks for clarifying Thomas!
Perfect timing given how many people are building with AI codegen tools today! Thanks for sharing the link.

Used this on CodeTV! Amazing tool

Maker

 📺 As Seen on CodeTV™ 🤣

Hunter

"look, mum, we were on CodeTV" 😸

123
Next