Four launches today are screens for watching agents work. That's a confession, not a category.
Cubicle, Agent Activity, Singularity and bmux are all, underneath the framing, a place to sit and watch agents do things.
That isn't a category forming, it's a symptom. You only build a cockpit when you don't trust the thing to fly itself. Nobody ever shipped a dashboard for watching their compiler.
The real problem is that agents fail quietly. They don't crash, they return something clean and plausible and slightly wrong, and the only way to catch it is to be looking at the moment it happens. So we build places to look.
I'd rather spend that effort making the failure loud. A run that stops and says I'm not confident about step 4 beats a beautiful live view of twelve agents, because the live view only works while someone is watching, and nobody is watching by week three.
Test I'd apply to any of these, mine included: if nobody opened the dashboard for a week, would you find out that something broke? If not, the dashboard isn't the product. It's the apology.
Replies
the "I'm not confident about step 4" framing assumes the agent knows when it's wrong, and that's the part I don't trust yet. at Dial the scary failures aren't the ones where the agent hesitates, they're the ones where it's completely fluent and wrong - it picked the wrong time slot and said it back in a perfectly confident sentence, nothing about the output looks uncertain. a dashboard would've shown a green checkmark the whole time. so I'd actually go further than your test: it's not just "would you find out if nobody opened it for a week," it's "does the agent itself know when it should be unsure," and most of them don't, they just have a tone of voice that always sounds equally sure of itself.
@galdayan Fair, asking a model how sure it is mostly gets you a confident answer about confidence. What I meant is cheaper than that, the stop has to come from outside the model. For your wrong time slot, the check isn't whether the agent hesitated, it's reading the slot back off the calendar after the write and comparing it to what the user asked for, which doesn't care how fluent the sentence was. Anything that can only be verified by the thing that produced it is the part I'd stop automating.
@asadmalik901 that framing holds up for a calendar write because there's a second source of truth sitting right there to read back. the case that worries me is the one without one - a phone call where the agent says something wrong to a human and hangs up. there's no system of record to diff against after the fact, the only trace is the transcript, which is the same fluent artifact that produced the error in the first place. the closest thing to an outside check we've found is cruder than I'd like: a second pass that reads the transcript against the original ask, not to judge tone, just to flag a handful of things that are checkable without trusting the first agent's self-report (did it quote a price, a date, a name back correctly). still inside the model, just not the same call, so it's a weaker guarantee than reading a calendar, but it's what's available when the real world doesn't keep its own log
@galdayan The price and the date and the name are the part that does have a record, they just don't live in the transcript. You know what you quoted and which slot you meant, so that check isn't really a model grading itself, it's a diff against something you already had, same as the calendar. Where I'd stop is everything else in the call, the judgement parts, because there's nothing to diff and a second pass there is just another opinion from the same place. So on a call I'd pick the handful of facts worth saying out loud and treat the rest as unverified, rather than trying to grade the whole thing.
@asadmalik901 yeah, that's a fair correction, I was lumping the checkable facts in with the judgment calls and they're not the same category. the original ask is already a record, so diffing a quoted price against it isn't the model grading itself, it's just reading back something that already existed outside the transcript. the judgment stuff is where I don't think a second pass buys you anything, since there's no outside thing to compare it to, just another generation from the same place. good way to put it