Our last launch answered a question we hadn't asked

We launched CoWork, a mobile app testing solution, on PH a while back and got a lot of useful feedback. Some of the most interesting comments came from developers who wanted to test their mobile apps before the merge, not after.

So we built QApilot MCP, that lets Claude, Cursor or Codex drive a real Android device. It launches Saturday.

I wanted to share the story of QApilot MCP. Here you go -

Where we started

Coding agents got good at writing mobile code. Verifying it didn't get easier. You can generate a checkout screen in twenty seconds and still not know whether checkout works. Your options are tapping through it yourself, or waiting for someone else to.

Point an agent at a phone and it sort of works. It taps by coordinates. Breaks on the first layout shift. Guesses at wait times. Produces tests that pass once. Anyone who's tried this has had the same afternoon.

Then the actual problems started

1. The UI tree is mostly noise

A real Android screen dumps hundreds of nodes. Most of them are layout wrappers with no text, no ID, and no purpose except nesting.

Feed that raw to a model and you burn context while burying the elements that matter.

So we don't. We cut the tree down before the agent ever sees it, and annotate what's left so a node knows where it sits relative to everything around it. That's what makes "the button next to the filter icon" resolvable at all.

2. "Checkout" is not an element

The agent says tap Checkout. Nothing on screen is called that. It might be a content-desc. Or text on a child node. Or an ID like btn_chk_cta. Or nothing at all until you scroll.

So we resolve it, through a few strategies in order of confidence, with fallbacks when the obvious ones miss. This is the piece that decides whether a run passes reliably, and it's unglamorous string and tree work rather than anything clever with a model.

3. Knowing when the screen is done

This is the one that surprised us. Fixed sleeps are why mobile tests are flaky. Too short and you assert against a half-drawn screen. Too long and every test crawls.

So we don't sleep. We watch for the screen to stop changing and move when it has. There's no sleep(2) anywhere in the executor.

Sounds trivial written down. It's most of the difference between tests that pass and tests that pass every time.

4. Knowing when to stop

An agent with a retry loop and no budget will burn twenty minutes and a lot of tokens looking productive.

So recovery is bounded. It retries, and when it's out of room it stops and reports what it couldn't do. Deciding to make the product give up was more contentious internally than anything else on this list.

5. Not re-learning the same app every session

Sessions are stateless and models forget. So we keep what worked. Each app builds up its own set of learned locators and flows, written only from runs that passed, and the agent gets handed those on the next launch.

Second run on the same app is meaningfully faster. Least difficult thing here to build, probably the most useful.

The split we landed on

Your agent reads the screen and writes the plan. Tap, type, check, in short batches. Our executor resolves each target against the live tree, acts, waits, asserts, recovers.

The agent decides what. We handle how. Every time we tried moving more of the "how" into the prompt, it got worse.

What we haven't solved

Setup. You need Node, Java, the Android SDK and Appium at pinned versions before anything runs. There's no way around it today.

If you already have an Android toolchain it's quick. If you don't, it's an afternoon. That's the next thing we're attacking.

Also: Android only, native apps only, local devices only. iOS is in progress.

Things I'd like to hear

If you test Android apps regularly: what do you still check by hand before every merge? Not the things you've automated. The things you haven't, and why.

And anything else you found interesting in this thread, I'm happy to discuss. I'm here in the comments all day.