Veil is an open-source Python web scraper that tries the cheapest method first and only escalates when a site blocks it so you're not running a full browser for every page.
⢠3 tiers: plain HTTP ā TLS impersonation ā stealth browser
⢠Resumable concurrent crawler (checkpoint, JSONL, dedupe)
⢠Proxy rotation, rate limiting, robots.txt by default
⢠Honest README: documents what it can't beat
Python, MIT, tested. A learning project built in the open.