AI that truly understands video. Uses multimodal models (Marengo/Pegasus) to search, analyze & generate text from video content at scale.
This is the 5th launch from TwelveLabs. View more
Jockey by TwelveLabs
Launching today
Jockey is the first AI that understands your entire media library just like you do, searching by person, moment, or context across every photo and video you've captured. Powered by TwelveLabs' advanced model stack, Jockey improves automatically with every update. Whether you need to connect via MCP for Claude/ChatGPT or build custom applications using our API, Jockey makes your media library instantly searchable and accessible.
Launch Team
TwelveLabs
I'm Aiden, co-founder and CTO at TwelveLabs. Super excited to share Jockey with you today.
Most media search is still metadata search: filenames, timestamps, maybe some object tags from an older computer vision model. None of that captures what's actually happening in your footage: who's in a scene, what they're doing, the context, dialogue, on-screen text. For that you need models that natively understand time and space in video, not a bag of sampled frames.
That's our stack. Marengo, our embedding model, resolves a query like "the moment we almost missed the flight" to real retrieval across video and images, not keyword matching. Pegasus, our video-language model, segments an entire video on a schema you define and returns structured, timestamped moments. Jockey is a unified agentic system that reasons across your videos and images: a reasoning model plus a memory layer that builds a knowledge store from your corpus, so it can decompose a query, retrieve, segment, and reason across the whole thing.
The point is a corpus-level understanding you can act on. Point Jockey at thousands of videos and images, say "cut me a highlight reel" or "pull the best viral moments," and it comes back with timestamped cuts you can use. The model-only approach can't do that as dumping one video into a context window is bound to a single file, and a single forward pass runs out of room fast. It can tell you about one video; it can't reason across your catalog or build a reel from thousands.
Because the models are what we ship and improve continuously, Jockey's reasoning and retrieval quality improve as we push new versions, meaning no re-integration on your end.
Two ways in:
MCP server: connect Jockey as a tool in Claude and query your library directly. ChatGPT coming soon.
API: full programmatic access to build custom retrieval or agent workflows on your own library.
This is a research preview, so if you hit edge cases (ambiguous queries, retrieval misses, latency) I want to hear about them. Let us know anytime!
Best,
Aiden
the corpus-level reasoning is the interesting jump here, most video search tools stop at "find the clip" and Jockey is going further to "reason across everything and build the reel." one thing I'm curious about that's different from the privacy/consistency questions above: as you keep adding new footage to a library over time, does Jockey only need to embed the new additions, or does growing the corpus mean periodically reprocessing the whole thing to keep the reasoning layer coherent?
TwelveLabs
Hey @galdayan, great question! Jockey only needs to embed the new additions, but it does it an intelligence manner to keep the entire reasoning layer coherent.
The compositional queries are where library-scale video search tends to break. Single-concept stuff like 'red car' works fine off embeddings, but 'the moment right after the door opens' needs temporal grounding that flat similarity search can't reach. When we built multimodal search over video the causal and ordering queries were exactly where recall fell off a cliff. Does Jockey's agent decompose those multi-step queries and reason over ordering, or is retrieval a single embedding lookup under the hood?
hi@dipankar_sarkar, great question. Through responses API (or query MCP tool), the agent decomposes the query into planned multi-steps to return the result. We separately offer a primitive knowledge-store search, which is the latter you asked "a single embedding lookup under the hood".
The "understands your whole media library" angle is really compelling. I already have product screenshots, launch videos, screen recordings, demos, and random clips scattered everywhere, and finding one specific moment usually depends on remembering the filename or roughly when it was created.
Searching by person, moment, or context feels much closer to how people actually remember media. connecting that through MCP is interesting too, because an agent could finally find the exact clip or screenshot needed for a task instead of asking me to dig through folders.. :) Curious how Jockey handles privacy and indexing for very large personal or company libraries.
Great name Aiden! The "no tags needed" pitch is the part I'd want to/ am going to stress test. Natural language search over video is great for discovery, but for compliance/legal use cases you usually want deterministic, auditable categories, not a model's best guess at a scene. Curious how consistent Pegasus's segmentation is run to run on the same clip.
would be cool if you added a built-in timeline view so you can jump straight to the exact moment a search result happens in the video. right now i think you only get timestamps in text, which is helpful but still requires manual scrubbing. basically a visual scrubber with the matched segments highlighted would save a lot of clicks.
honestly the search is way sharper than i expected, like i threw in a random cooking video and it pulled out the exact moment they mentioned "fold in the eggs" without any tagging. pretty cool to actually see video understanding feel useful