Launched this week
Webpage Text Extractor
Extract clean text and Markdown from public web pages
16 followers
Extract clean text and Markdown from public web pages
16 followers
Turn public web pages into clean text, article text, or Markdown for LLM prompts, RAG inputs, content review, and exports. Add URLs, choose the text shape, and get page text with metadata, headings, links, counts, redirects, and timestamps.
How does the extractor handle pages behind paywalls or ones that require JS rendering to show the actual content?
How does it handle pages that are heavily JavaScript-rendered or gated behind a login, and is there an API I can hit so I can pipe results straight into my own scripts?
How does it handle pages that block scraping or require login, and does it respect robots.txt under the hood?
how does it handle pages that block scrapers or require login, is there a way to pass cookies or proxy through when a site throws a 403?
Tried it on a couple of long reads and the Markdown output came back really clean, links and headings preserved nicely. The redirect flag was a nice touch, saved me from chasing broken sources.
Pasted in a few long articles and the Markdown output kept headings and links intact, which saved me a bunch of cleanup before feeding it into a prompt. The redirect and timestamp metadata was a nice touch I didn't expect.