VMTP

Video understanding for LLMs

130 followers

Video understanding for LLMs

130 followers

Visit website

Unified API

•

AI Infrastructure Tools

•

Cloud Computing Platforms

VMTP or Visual Media Transcription Protocol is a video processing protocol for LLMs. Using this, you can let your LLMs & AI agents understand video input.

Free

Launch tags:API•Developer Tools•Artificial Intelligence

Launch Team

Framer AI AgentsDesign and publish professional sites with AI

Promoted

Maker

📌

Hello PH community 👋 Udit this side from VMTP team. We see a lot of you guys building amazing AI agents and shipping it on ProductHunt. LLMs can getting crazy powerful and multi-modal - text, audio, image, voice, you can process all of them. However, most LLMs can't still process videos. And we are solving this exact problem with VMTP. VMTP is a video-processing protocol - you can use it build amazing AI agents which can understand videos. The best part? It supports over 100+ AI models (thanks to an internal Openrouter integration). This is an early preview and pre-release of VMTP. I hope you like this. Please share your feedback. :) Regards Udit

Report

1yr ago

@udit_exthalpy A couple of questions came to mind, How does VMTP handle complex tasks like object tracking, scene segmentation, or audio-visual synchronization? + And for developers integrating this into existing frameworks, are there specific hardware or software requirements to ensure smooth performance?

Report

1yr ago

Maker

@yatheen_brahma Hey Yatheen, VMTP creates multiple time-stamped chunks of both audio and video files. This way, audio-visual synchronisation stays aligned. Now, about object tracking, we are working on it. This version is currently deployed on api so there is no such hardware or software requirements too. :) Feel free to reach me out and if you plan to contribute.

Report

1yr ago

Maker

@sonu_goswami2 Thanks Sonu

Report

1yr ago

Maker

If anyone interested in our subReddit for more discussions and tutorials, here it is --> https://www.reddit.com/r/Deeptrain/

Report

1yr ago

minimalist phone: reduce your screentime

💎 Pixel perfection

I collect these YouTube tools – thank you for sharing! Is there any way how it could help me to create better videos? As I understood, it analyses existing content. What about recommendations for improving my upcoming content? Does it do the work?

Report

1yr ago

Maker

@busmark_w_nika This can be a very strong use case. It can seamlessly see and analyse videos - both yours' and your competitors'. So, yes, it can tell you and give exact insights over where to improve. I will surely post more about it.

Report

1yr ago

minimalist phone: reduce your screentime

@udit_exthalpy Would like to see more insights in this project :)

Report

1yr ago

Maker

@busmark_w_nika Please follow my substack then at uditraj.substack.com .. It is a small place I get o write freely about things I do.

Report

1yr ago

🔌 Plugged in

Just to confirm: VMTP basically takes a video URL and turns it into an AI-friendly transcript, right? I also have a few other questions: 1. Does it currently work with YouTube and Vimeo? Are there plans to add other video platforms later? 2. When can we expect an API to be available, and what will the pricing structure look like? Anyway, congrats on your launch, Udit!

Report

1yr ago

Maker

@bonvisions The base API works to convert local videos into transcripts. It simply breaks the video into an audio and a visual part and then chunk it down into multiple smaller pieces. Hence, tells AI what the video is about. Coming to your questions, 1. We are building connectors (as shown in demo). This will allow you to import videos from YT, Vimeo, etc. 2. I am alone working on this project. Tbh, I will soon ship its api and pricing will be something like a recharge and pay-as-you-go plan. (Will surely need support of early supporters). Thanks Bon for your support!

Report

1yr ago

love this! Do u have a twitter? Would love to send u a dm to talk about some business opportunities on this

Report

1yr ago

Maker

@noah_solomon1 Yes Noah, It's @AkhouriUdit Please DM

Report

1yr ago

Hi! I just started Search Riot YouTube channel and would love to do a deep dive for a video on how businesses could take advantage of this! Joined the waitlist with X handle: Search_Riot Hope to test drive soon! @udit_exthalpy 🚀

Report

1yr ago

Maker

@daviddennison Definitely! Would love to work with you :)

Report

1yr ago

I had submitted the form before but did it again just in case!

Report

1yr ago

🔌 Plugged in

if you love making videos then its a best tool for you

Report

1yr ago

Maker

@burnscolem4696 Exactly.. This is a perfect tool for people experimenting with videos + LLMs

Report

1yr ago

Reviews