How do you test the "book a flight to London" agent workflow
How do you test a workflow that spans four services?
Been hitting the same wall for a while and curious how others handle it.
What's the failure shape?
Someone books a flight to London. Four things have to happen: confirm the booking, send a Slack message, fire a confirmation email, block the calendar. Travel is one vendor, messaging is another, email is another, calendar is another.
We ran two versions against the same request. The second sends the right message, fires the right email, confirms the booking. Every response is a 200. What it never does is block the calendar.
Nothing looks wrong. The flight is booked. The user got a message and an email. They find out at 6am when they double-booked the same slot.
Why is this hard to catch?
A version can pass the call log check, pass the write check, and still fail the only question that matters. Is that slot actually blocked? Those are four separate questions and they give different answers.
What actually helped?
Re-running from identical state mattered more than I expected. Otherwise you're comparing two runs that started differently and calling it a diff. We also needed the services to behave like prod, 429s, auth failures, delayed webhooks, duplicate deliveries, without real prod keys. That's what FetchSandbox gives us: service twins that surface the failure modes the happy path never hits.
Is anyone actually testing these flows end to end, or is it mostly per-service mocks and hope?
Replies
I would test this as a state machine, not as four successful API calls. Give the workflow one correlation ID and assert the final business invariant: exactly one confirmed booking, one calendar hold covering the same interval, one notification, and one email. Then run a fault matrix from an identical seed: each service times out before and after committing, returns 429, delivers a webhook late or twice, and becomes unavailable during compensation. The important case is the ambiguous timeout—if the calendar write succeeded but the response was lost, a retry must query by idempotency key instead of creating another hold. Record every state transition and make the test fail on an orphaned booking or hold, not just a non-200. I would also add a reconciliation test that detects and repairs partial completion after the workflow process itself restarts.