I've been designing a voice pipeline lately, and the whole design boils down to one idea: a dual-speed loop.
Fast loop: a multimodal decision model whose only job is to hold the conversational rhythm. Within ~200ms of you finishing a sentence, it responds whether that's answering, asking a follow-up, or just saying "hmm, let me check" is decided in real time.
Slow loop: a voice agent doing the actual work in the background searching, running tasks, anywhere from seconds to minutes. When it's done, it picks the right moment to jump back into the conversation, guaranteeing depth and quality.
