AI that truly understands video. Uses multimodal models (Marengo/Pegasus) to search, analyze & generate text from video content at scale.
This is the 5th launch from TwelveLabs. View more
Jockey by TwelveLabs
Launching today
Jockey is the first AI that understands your entire media library just like you do, searching by person, moment, or context across every photo and video you've captured. Powered by TwelveLabs' advanced model stack, Jockey improves automatically with every update. Whether you need to connect via MCP for Claude/ChatGPT or build custom applications using our API, Jockey makes your media library instantly searchable and accessible.
Launch Team
TwelveLabs
I'm Aiden, co-founder and CTO at TwelveLabs. Super excited to share Jockey with you today.
Most media search is still metadata search: filenames, timestamps, maybe some object tags from an older computer vision model. None of that captures what's actually happening in your footage: who's in a scene, what they're doing, the context, dialogue, on-screen text. For that you need models that natively understand time and space in video, not a bag of sampled frames.
That's our stack. Marengo, our embedding model, resolves a query like "the moment we almost missed the flight" to real retrieval across video and images, not keyword matching. Pegasus, our video-language model, segments an entire video on a schema you define and returns structured, timestamped moments. Jockey is a unified agentic system that reasons across your videos and images: a reasoning model plus a memory layer that builds a knowledge store from your corpus, so it can decompose a query, retrieve, segment, and reason across the whole thing.
The point is a corpus-level understanding you can act on. Point Jockey at thousands of videos and images, say "cut me a highlight reel" or "pull the best viral moments," and it comes back with timestamped cuts you can use. The model-only approach can't do that as dumping one video into a context window is bound to a single file, and a single forward pass runs out of room fast. It can tell you about one video; it can't reason across your catalog or build a reel from thousands.
Because the models are what we ship and improve continuously, Jockey's reasoning and retrieval quality improve as we push new versions, meaning no re-integration on your end.
Two ways in:
MCP server: connect Jockey as a tool in Claude and query your library directly. ChatGPT coming soon.
API: full programmatic access to build custom retrieval or agent workflows on your own library.
This is a research preview, so if you hit edge cases (ambiguous queries, retrieval misses, latency) I want to hear about them. Let us know anytime!
Best,
Aiden
@aiden_lee7 Congrats on the launch, Aiden. For teams using Jockey at scale, what’s been the biggest real-world surprise when the system reasons across thousands of videos? Any specific edge cases you didn’t expect that others should watch out for?
the corpus-level reasoning is the interesting jump here, most video search tools stop at "find the clip" and Jockey is going further to "reason across everything and build the reel." one thing I'm curious about that's different from the privacy/consistency questions above: as you keep adding new footage to a library over time, does Jockey only need to embed the new additions, or does growing the corpus mean periodically reprocessing the whole thing to keep the reasoning layer coherent?
TwelveLabs
Hey @galdayan, great question! Jockey only needs to embed the new additions, but it does it an intelligence manner to keep the entire reasoning layer coherent.
The compositional queries are where library-scale video search tends to break. Single-concept stuff like 'red car' works fine off embeddings, but 'the moment right after the door opens' needs temporal grounding that flat similarity search can't reach. When we built multimodal search over video the causal and ordering queries were exactly where recall fell off a cliff. Does Jockey's agent decompose those multi-step queries and reason over ordering, or is retrieval a single embedding lookup under the hood?
hi@dipankar_sarkar, great question. Through responses API (or query MCP tool), the agent decomposes the query into planned multi-steps to return the result. We separately offer a primitive knowledge-store search, which is the latter you asked "a single embedding lookup under the hood".
Great name Aiden! The "no tags needed" pitch is the part I'd want to/ am going to stress test. Natural language search over video is great for discovery, but for compliance/legal use cases you usually want deterministic, auditable categories, not a model's best guess at a scene. Curious how consistent Pegasus's segmentation is run to run on the same clip.
would be cool if you added a built-in timeline view so you can jump straight to the exact moment a search result happens in the video. right now i think you only get timestamps in text, which is helpful but still requires manual scrubbing. basically a visual scrubber with the matched segments highlighted would save a lot of clicks.
@buketmdrp to your point, the existing LLM convention of text responses is often insufficient, especially when working with video. this is exactly why we built the MCP! as part of the tool set, we've added the ability for Jockey to render a response in a format much more suited for video. so instead of getting back text and timestamps, you can actually visualize with playback the exact moment that Jockey is referring to. add it to your claude as a connector (https://mcp.twelvelabs.io/jockey/mcp) and let us know if you have any feedback!
The "understands your whole media library" angle is really compelling. I already have product screenshots, launch videos, screen recordings, demos, and random clips scattered everywhere, and finding one specific moment usually depends on remembering the filename or roughly when it was created.
Searching by person, moment, or context feels much closer to how people actually remember media. connecting that through MCP is interesting too, because an agent could finally find the exact clip or screenshot needed for a task instead of asking me to dig through folders.. :) Curious how Jockey handles privacy and indexing for very large personal or company libraries.
Would love a timeline-based annotation view where I can click any moment in a search result and instantly see the surrounding visual and audio context. Right now results feel like a black box of timestamps, but seeing a quick visual snapshot before clicking through would make reviewing long footage way faster.
@pek4yco thanks for the feedback! would love for you to use the MCP (https://mcp.twelvelabs.io/jockey/mcp) which has a "render" tool that visualizes responses in a much more video-friendly way. instead of a block of text with timestamps, you can see results with playback.