Vedha
Playwright tests that fail on silent 500s
Start new thread

Vedha - Playwright tests that fail on silent 500s

by•
Change one import and your existing Playwright tests also fail when the server returned a 5xx or a write failed, even though the page said "Saved". No rewrite, no new assertions. Vedha also ships an MCP server, CLI and SDK so AI agents check the server, not just the screen: each action is verified against the page, the network and a read-back of stored data. Open source, Apache-2.0.

Add a comment

Replies

Best
Maker
📌

Hi Product Hunt, I'm Arun.

I kept seeing end-to-end tests go green while the feature was broken: the page shows "Saved", the API returned a 500, nothing was stored. Playwright catches that only if you wrote a network assertion for that exact request, and most tests don't.

Vedha is built on Playwright. Two ways to use it:

  • Existing suites: change @playwright/test to @vedhajs/playwright-test. Tests run as before, and a test that passes its own checks still fails if the app answered with a 5xx or a write failed outright.
  • AI agents: an MCP server where each action is checked against the page, the network and, for writes, a read-back of the stored data. On held-out tasks written before any run, the agent caught 41 seeded defects vs 24 with Playwright MCP, using 12-46% fewer tokens on healthy tasks.

Honest limits: the agent numbers come from one free model at 3 trials per cell; when an expert hand-writes every check, Vedha and Playwright tie; guard mode can't see a write that returns 200 and stores nothing. All raw data is in the repo.

I'd love to hear where it gets in your way.