Subscribe
Sign in
Webᵀ Crawl by Web Transpose
Realtime web data for LLMs.
Start new thread
Mike

Webᵀ Crawl by Web Transpose - Realtime web data for LLMs

by
✨ Turn full websites into datasets for building custom LLMs with Webᵀ Crawl. Give us just 1️⃣ URL and let Webᵀ Crawl handle the rest. 💻 Quickly turn full websites content (like PDFs, FAQ, etc.) into [prompts for fine-tuning] and [chunks for vector databases].

Add a comment

Replies

Best
Mike
Maker
📌
Hey everyone! I'm Mike – I'm working on 🟪 Web Transpose, a collection of tools to make the web yours. → https://webtranspose.com/ I'm excited to share my first tool – 🟪 Webᵀ Crawl. I had quite a few friends building chatbots from websites so I decided to build a tool to streamline the process. I also made a few tutorials about finetuning and vector databases for LLMs here: → https://www.webtranspose.com/blog 🙆‍♂️ I'd love for you to check 🟪 Webᵀ Crawl out and have a play. Let me know what you end up building. Would love all of your feedback. I'm excited to chat about all things web data / web scraping / web crawling in the comments! orthogonally ❤️ Mike mike@webtranspose.com
Aditya Saxena
Hey Mike, this is a really cool tool - do you plans for an API too? I'd like to think that it'd help many devs including me who want to automate the chatbot-building process :)
Mike
Maker
@asaxena1415 Absolutely! API coming soon.
Mike
Maker
@asaxena1415 Thanks for checking out Webᵀ Crawl!
Yash
Awesome, Mike!
Mike
Maker