AI that truly understands video. Uses multimodal models (Marengo/Pegasus) to search, analyze & generate text from video content at scale.
This is the 7th launch from TwelveLabs. View more
Pegasus 1.6 by TwelveLabs
Launched this week
Pegasus 1.6 is the TwelveLabs model that understands video from a first-person point of view, built for the egocentric and teleoperated footage robotics and physical AI teams already collect. It recognizes entities more accurately, and produces persistent metadata across segment types. Point it at raw teleoperation or wearable footage and get back labeled, timestamped, robot-ready data. No manual labeling pass required.
Launch Team
World model AI : Sharm SEKAISimulate your idea on a living world.
Promoted
Go pegasus1.6!
let's gogogog!
@tessak22 you're the best, Tessa! thanks for the continuous support
Turning raw teleoperation and wearable footage into labeled, timestamped robot training data solves a bottleneck that physical AI teams feel constantly, since collecting the video is easy and annotating it is not. Understanding first person perspective specifically is the right specialization. Do the persistent metadata labels stay consistent across long recording sessions?
Spot on. IMHO this could be a game changer for robotics and physical AI teams.
@karimbenkeroum
Hi Kareem, great question! Pegasus supports Time Based Metadata mode which allows for fine atomic action segmentation on long videos. Each segment is then labeled for fields such as action description, hands used, action per hand, tool used, and so on. So you can see a high quality dense captioning output that is perfect for downstream physical AI tasks!
oh yes, and try it for yourself on the playground at twelvelabs.io
"no manual labeling pass required" is the line that gives me pause more than it reassures me. for next-action prediction specifically, a mislabeled hand-object interaction isn't a cosmetic error, it's training signal a downstream policy will actually learn from, and without a human check somewhere the model's blind spots just become the robot's blind spots, quietly and at scale. is there any sampled human QA on the output metadata, or is the pitch literally zero review from raw footage to labeled training data
turning teleop footage into labeled robot data without manual work is huge 🤖 which robotics teams are using it so far?