Advice Needed - Web Scraping
We have a lot of app developers on here so I am hoping to bend an ear on a solution. Is it possible for a web scraper to bypass Login? And how would you go about it? The web scraper we have is able to scrape a front end of a website but at the moment we are struggling to get it to scrape a website that requires you to register to get access to the information we are wanting to scrape. For example, we can scrape the prices of products if they are visible to the public, but unable to do it if you have to register to the site first to see the products. Signing up is not an issue, its getting the scraper to then work using those registered details to scrape. Any tips or advice would be excellent. Or if there is a tool that does that kind of thing you could highlight to me that would be great. Many thanks.