What if latency were no longer the bottleneck — what would voice actually unlock?
I've been designing a voice pipeline lately, and the whole design boils down to one idea: a dual-speed loop.
Fast loop: a multimodal decision model whose only job is to hold the conversational rhythm. Within ~200ms of you finishing a sentence, it responds — whether that's answering, asking a follow-up, or just saying "hmm, let me check" is decided in real time.
Slow loop: a voice agent doing the actual work in the background — searching, running tasks, anywhere from seconds to minutes. When it's done, it picks the right moment to jump back into the conversation, guaranteeing depth and quality.
Bridging them is a global context controller: it syncs context between the two loops and decides when slow-loop results get injected — and whether it's worth interrupting the current turn.
The inspiration is basically a human secretary: acknowledge you first, go do the thing, come back and cut in when it's done. Humans spent millennia evolving this "social protocol for masking latency" — I just engineered it.
Once it worked, the improvement in conversational naturalness was obvious: waiting became part of the conversation for the first time, instead of awkward dead air.
But here's where I'm stuck: beyond "book my flight without making me wait in silence," what else is this actually for?
A few directions I've thought of, none of which I'm sure about:
Kitchen: fast loop chats with you, slow loop tracks "is the water boiling, is the oven done" — and interrupts you when it's time
Long-term companionship: slow loop observes across days — "he's mentioned knee pain three times this week" — and speaks up proactively, instead of waiting to be asked
Practice partner: fast loop plays the counterpart, slow loop acts as the judge, cutting in to correct you at the right moment
The common thread seems to be: conversation is just the interface — the real value happens continuously outside of it. Conversely, pure chitchat doesn't need dual-speed at all. That would be over-engineering.
So I'd love to hear from you:
What moments in your life are "I'm busy with A, but wish someone were watching B"?
AI proactively interrupting you — when does it feel great, and when does it make you want to throw your earbuds?
Or am I thinking about this backwards — is the killer use case for dual-speed actually inside casual conversation?
Replies