hey PH 👋 i'm still pretty new to python and built this mostly to learn, so honestly any feedback helps. the idea: most scrapers either send a plain http request and get blocked, or spin up a whole headless browser for every page (slow + expensive). veil tries the cheap stuff first and only moves up when it actually gets blocked: plain http → TLS fingerprint impersonation → stealth browser it's also got a crawler that resumes if it crashes, rotates proxies, and respects robots.txt by default. the part i'm actually proud of: i tested it against real sites and wrote down what it CAN'T do. it gets past JS challenges, but if a site throws a 403 before any javascript runs, that's an IP reputation thing and no code fixes it — you just need residential proxies. that's in the readme, i didn't wanna pretend it beats everything. it's open source (MIT), and there's a CONTRIBUTING guide + good-first-issues if anyone wants to build with me. would love to hear what i got wrong 🙏

Veil

A web scraper that uses the lightest method that works

A web scraper that uses the lightest method that works