Muse Realtime Avatar - Turn voice chats with Muse into live video calls
by•
Muse Realtime Avatar gives Meta's Muse agent a face you can video chat with. Voice and video come from one speech stream, so lips, expressions, gestures and posture move with every word, and the reply starts in about 870 ms. Unlike talking-head avatars that only animate a face, it moves hands and full body too, and the character stays consistent from turn to turn. In Meta's live-call tests, raters preferred it 78% of the time over Runway Characters and 88% over HeyGen LiveAvatar.
Replies
Hey PH folks!
Meta just gave its Muse agent an actual face. You talk, it talks back on video, and the eyebrows, hands and posture move with what it's saying. Way past the "mouth flapping on a still photo" stage 🌚
The nerdy bit I loved: the big teacher model needed 120 passes for every video chunk. The live one needs 2. That's how they get the reply started in under a second.
Heads up, it's rolling out in the Muse app in the US for 18+, so some of you will get it before others. Europe, we need to wait a bit as ususally.
Check out more and see it in action here https://research.meta.ai/blog/bringing-your-muse-to-life
Moving the hands and full body rather than just the face, and keeping the character consistent turn to turn, is what would stop me noticing it is an avatar after a minute of talking.
The full-body part is what caught me — most avatars fall apart for me once the hands start moving, so keeping the character consistent turn to turn matters more than the face. Curious how it holds up over a longer call though: does the character stay stable through, say, an hour of back-and-forth, or does it start drifting the longer the conversation runs?