Product Mentions - Aggregating all Amazon products linked from Reddit

@rrhoover Hey Ryan! So turns out I'm the creator of this site. For some reason I never put google analytics on the site so didn't see people viewing it. I searched google this morning for "product mentions" to see the site was at the top of google, and turns out this thread is even higher than the site! When I made the site it was super simple in terms of what I was looking for, and since I deployed it back in December, I haven't changed the code at all. I'm curious if you have suggestions on ways to improve it or expand it's capabilities. If not, I'm still super excited to see this and people liked it.

@jack_schultz nice job, Jack! How are you crawling reddit and matching mentions of a particular product with its Amazon link?

@rrhoover Basically have a python scraper that every minute hits Reddit's all comments json url and scrapes the comments and uses a regex to see if an amazon url is in the text. From there, use the Amazon API to search the Amazon ID and if it returns a response saying that product exists, grab the info and image they have and save all of that in the DB. Now obviously the one concern is whether the scraping of Reddit comments is legal and allowed by Reddit, but the main thing here is that I'm very slowly hitting their servers and definitely not causing issues on that front. But again, still not sure if that's allowed by them.