Hey Product Hunt 👋 We're Eric, Caleb, and Nick from Firecrawl. Today we're launching /monitor, the easiest way to keep your AI agent in sync with the web.
We built /monitor because we kept hearing the same thing. A lot of our customers were already using Firecrawl to watch specific pages, re-scraping the same pricing pages, docs, changelogs, and filings on a loop just to catch when something changed. It makes a ton of sense, but doing it by hand means you either over-poll and burn tokens on pages that didn't change, or under-poll and miss the update that mattered.
So we turned it into a product. Point it at a URL, describe what to track in plain English, and Firecrawl checks the page on your cadence, compares it to the last version, and pings your agent over webhook the moment something meaningful changes. Your agent only ingests what actually changed, so you can cut token usage by up to 90%.
There's nothing to wire up yourself. The schema, scheduling, diffing, and delivery are all handled for you, and you see the estimated monthly cost before you flip a monitor on. Changes arrive by signed webhook or email, with a permalink for every diff you can hand straight to another agent. It runs on Firecrawl's /scrape under the hood, so JS-heavy pages get tracked reliably too.
If you've got an agent re-scraping the same docs, changelogs, or competitor pages on a loop, this one's for you.
You can try it out here: https://docs.firecrawl.dev/featu...
Would love to hear what you think.
Report
@ericciarla The over-poll versus under-poll tradeoff is exactly the tax nobody budgets for when they wire this up by hand. Pricing the diff before you flip a monitor on is the detail I would not have thought to ask for, but immediately want. One question: when a page changes layout but not meaning, does the diff stay quiet, or does my agent get woken up for a CSS refactor?
Report
@ericciarla Great idea, i especially like the part where the agent ingests only what changed. How are you presenting this delta though ? Like if I change the layout but keep the content same, will it detect as a change or not ?
Report
@ericciarla Really like the webhook diff approach. How do you choose poll frequency without overwhelming downstream systems?
Love this build. The 90% token cut is wild. That's the number every team running AI products is quietly desperate for.
We pull buyer-side data into FireCoach all day long for sales prospect profiles, and the re-scrape vs. monitor tradeoff is exactly the thing we keep fighting. Going to share this with our engineering team.
Quick question: does the diff get summarized in natural language before hitting the webhook, or does the receiving agent still have to interpret the raw diff?
Either way, this is one of those "obvious in hindsight" products. Nice work!
@midori_verity That's incredible. Thank you! Right now it's just the raw diff and the receiving agent has to interpret that raw diff, which they are pretty good at.
@naimz Yeah, so it actually depends on the goal you set. If you set a specific goal about a specific selector, then it will only notify you around that. Vice versa for entire pages!
@jalcantara Sounds amazing. Let me know how you like it!
Report
Been waiting for something like this. Setting up custom polling logic to watch external data sources is one of those tasks that eats 2-3 hours and nobody talks about
Been doing the "scrape on a cron + diff manually" thing for competitor tracking for months. The webhook approach is cleaner. One question/ does it handle pages that hydrate content via JS after first render?
Report
the 90% token reduction is the part i wish more people talked about. running webhooks on diffs instead of full crawls is what makes long running agents actually viable in production. curious how you handle noisy DOM changes (analytics scripts, ad slots, dynamic timestamps) before triggering the webhook. is that filtered at the diff layer, or is the agent on the receiving end expected to sort signal from noise?
Report
agents re scraping entire pages every hour just to check if one price changed is such a waste of tokens and money. only ingesting the diff is how it should've always worked
This is one of those features that seems obvious only after someone builds it. Monitoring changes instead of constantly re-scraping pages feels like a much smarter approach. Congrats on the launch!
The shift from "blind polling" to "intelligent diffing" is exactly what dev teams need to keep AI agent costs under control. I’ve lost count of how many tokens I’ve burned re-scraping unchanged docs.
How does /monitor handle dynamic content (like JS-heavy dashboards) vs. static text? Does it ignore irrelevant UI changes (like ad rotations or timestamps) to ensure the webhook only fires for meaningful data shifts?
🕸️
Report
How does the tool bypass advanced anti-scraping blocks like Cloudflare or CAPTCHAs?
Can the system detect and highlight a single word change within a text?
Replies
Firecrawl
@ericciarla The over-poll versus under-poll tradeoff is exactly the tax nobody budgets for when they wire this up by hand. Pricing the diff before you flip a monitor on is the detail I would not have thought to ask for, but immediately want. One question: when a page changes layout but not meaning, does the diff stay quiet, or does my agent get woken up for a CSS refactor?
@ericciarla Great idea, i especially like the part where the agent ingests only what changed. How are you presenting this delta though ? Like if I change the layout but keep the content same, will it detect as a change or not ?
@ericciarla Really like the webhook diff approach. How do you choose poll frequency without overwhelming downstream systems?
Firecoach AI
Congrats Eric, Caleb, and Nick! 🔥
Love this build. The 90% token cut is wild. That's the number every team running AI products is quietly desperate for.
We pull buyer-side data into FireCoach all day long for sales prospect profiles, and the re-scrape vs. monitor tradeoff is exactly the thing we keep fighting. Going to share this with our engineering team.
Quick question: does the diff get summarized in natural language before hitting the webhook, or does the receiving agent still have to interpret the raw diff?
Either way, this is one of those "obvious in hindsight" products. Nice work!
Firecrawl
@midori_verity That's incredible. Thank you! Right now it's just the raw diff and the receiving agent has to interpret that raw diff, which they are pretty good at.
Mailwarm
Can you set rules for what counts as a meaningful change, like only when certain sections or selectors update?
Firecrawl
@naimz Yeah, so it actually depends on the goal you set. If you set a specific goal about a specific selector, then it will only notify you around that. Vice versa for entire pages!
Mailwarm
@ericciarla Grand, thanks
Firecrawl
@jalcantara Sounds amazing. Let me know how you like it!
Been waiting for something like this. Setting up custom polling logic to watch external data sources is one of those tasks that eats 2-3 hours and nobody talks about
Been doing the "scrape on a cron + diff manually" thing for competitor tracking for months. The webhook approach is cleaner. One question/ does it handle pages that hydrate content via JS after first render?
the 90% token reduction is the part i wish more people talked about. running webhooks on diffs instead of full crawls is what makes long running agents actually viable in production. curious how you handle noisy DOM changes (analytics scripts, ad slots, dynamic timestamps) before triggering the webhook. is that filtered at the diff layer, or is the agent on the receiving end expected to sort signal from noise?
agents re scraping entire pages every hour just to check if one price changed is such a waste of tokens and money. only ingesting the diff is how it should've always worked
Firecrawl
@tina_chhabra 100% this is overdue!
Stripo.email
This is one of those features that seems obvious only after someone builds it. Monitoring changes instead of constantly re-scraping pages feels like a much smarter approach. Congrats on the launch!
@ericciarla @caleb_peffer @nickscamara Congrats on launching /monitoring!
The shift from "blind polling" to "intelligent diffing" is exactly what dev teams need to keep AI agent costs under control. I’ve lost count of how many tokens I’ve burned re-scraping unchanged docs.
How does /monitor handle dynamic content (like JS-heavy dashboards) vs. static text? Does it ignore irrelevant UI changes (like ad rotations or timestamps) to ensure the webhook only fires for meaningful data shifts?
🕸️
How does the tool bypass advanced anti-scraping blocks like Cloudflare or CAPTCHAs?
Can the system detect and highlight a single word change within a text?