Framer AI AgentsDesign and publish professional sites with AI
TryCase
Disposable test environments for AI coding agents
292 followers
Disposable test environments for AI coding agents
292 followers
TryCase gives AI coding agents disposable Linux environments to run apps, test changes end to end, capture screenshots and recordings, and return verified code instead of asking you to test manually.
Free Options
Launch Team
The 'an agent handles verification' step is where I've watched this quietly break. When the same model family does the work and the check, the verifier tends to trust the doer's framing of what success looks like, so it happily confirms a screenshot of the wrong screen. What helped me was feeding the verification agent only the original task spec plus the artifact, never the doer's transcript, so it can't inherit the optimistic story of what happened. Does TryCase hand the checker the full run log, or just the recording and a fresh prompt?
@dipankar_sarkar That’s a really useful observation. Today TryCase does not have its own checker, so the external agent decides what context the verifier receives.
I’m considering a more opinionated workflow where TryCase connects to GitHub, generates QA scenarios and temporary E2E tests for each PR, and runs them. A separate agent would then review the results, screenshots, and recordings before reporting issues.
What context would you give that reviewing agent? Would you include the PR description, code diff, generated tests, and artifacts while hiding the execution transcript, or restrict it to the expected behavior and artifacts only?
Congrats Ben — the part that resonates is “verified code instead of asking you to test manually.” In my own coding-agent loop, the weak point is not generation, it's proof quality: agents say “done” after lint passes, but the actual product flow is still broken.
A small thing I’d love to see in TryCase is a compact verification receipt: commands run, browser path tested, screenshots/video links, and what failed before the final pass. That would make it much easier to trust the result or review it later.
Curious if you're thinking about a standard “done packet” that agents can return to humans or CI?
@grace_lee26 Thanks Grace, this is close to what I’m thinking about for the next version. TryCase would connect to GitHub and QA each PR by generating scenarios and temporary E2E tests, running them, and having a separate agent review the results, screenshots, and recordings.
I’m still working out what the final receipt should contain and where it should live. Would you prefer a GitHub check summary, a PR comment, or a separate report? What is the minimum evidence you would need before trusting it?
The 'come back with proof instead of just code' framing is the right one, and the disposable-env angle solves a real mess. The gap I keep hitting past this: a run passing proves it ran, not that it did the right thing. The last mile is a pass criterion the agent doesn't get to write for itself, otherwise 'verified' quietly means 'it didn't crash.'
@theuniverseson You’re right. Today TryCase provides the isolated environment and tools for an agent to run the app, execute tests, exercise browser flows, and collect evidence. It does not yet independently define the pass criteria or verify that the agent chose the right criteria. That still comes from the agent and the user’s instructions.
I’m considering a more opinionated PR-based QA layer with separate agents for generating scenarios and reviewing the evidence, but that is not built yet. In your ideal workflow, where should the source of truth for passing come from? The PR description, human-written acceptance criteria, existing tests, or historical product behavior?
finally something that lets my agent actually run the code instead of me playing QA. tried it on a small flask app and got back screenshots plus a clean diff, which honestly saved me a whole round trip.
@serpiltrmey8sl That’s really good to hear, thanks for trying it. Which coding agent did you use, and did it get the Flask app running from one prompt or need some steering? I’d also love to know which part of the setup felt least smooth.
This maps to the messy part of agent coding for me: not the patch itself, but proving it ran in a clean environment. The useful constraint is keeping the proof lightweight enough that agents actually include it every time.
@jimmy_lee12 That’s a good point. Today the external agent decides what evidence to capture, so the amount can vary quite a bit.
I’m working on PR-based QA now, and I want the proof to be consistent without becoming noisy. What would feel like the minimum useful receipt for each PR? A short pass or fail summary with commands and one screenshot, with recordings only when something fails?
screenshots are harder to fake than a transcript — but a vision model skimming the recording still reads 'done' off surface cues. a success toast looks identical whether the write landed or silently failed. the second agent relocates the trust, doesn't close it.
@qifengzheng You’re right. Today TryCase can provide the environment and evidence, but it does not close that trust gap by itself. A second agent reviewing screenshots or recordings can still approve the wrong thing.
I’m working on PR-based QA where the tests should also verify underlying state through API responses, database assertions, logs, or events. But the generated assertions can still be weak. Where would you anchor the pass criteria? Human-written acceptance criteria, existing tests and contracts, or a human review of the generated scenarios before they run?
@ben_chomsang tests + contracts are the honest anchor — post-conditions the model can't retrofit. anything a human still has to eyeball just moves the trust seam without closing it. gen scenarios need the same discipline or slide back into surface cues.
the disposable Linux environment idea is such a clean solve for the "I cant actually verify this" problem with coding agents. nice execution.
@necatisere96416 Thanks Necati. What coding agent are you using today, and which parts of its work do you still find yourself verifying manually?