Viktor.comAn AI coworker that actually does the work
Promoted
Launched this week
Extract, crawl and structure web data at scale with Olostep's Web Data API. Built for AI teams, data pipelines and automation. Fast, reliable and cost-effective.
Forum Threads
Olostep: Web Data Infrastructure for AI
Welcome to Olostep. We're building the infrastructure AI agents need to do real work on the web.
The web is the backbone of every business and the home of humanity's collective knowledge. Humans browsed it manually for decades. Now Agents are knocking on the door.
In late 2025, with improvements in AI models, we entered an era where software stopped following instructions and started thinking. We shifted from an instruction-based world to an intent-based one. The agentic era. That shift breaks a lot of infrastructure that quietly assumed a human in the loop: email, payments, identity, search.
Search asked you to do the rest. You typed a query, got some links back, and then opened the tabs, read, compared, and pieced it together yourself. In an agentic search world, what comes back isn't links, and isn't an answer. It's the work, done: hundreds of sources searched, pages read, fields extracted, conflicts resolved, output structured and delivered.
The web's second user is arriving, at machine scale. Everything it needs to work has to be built again. The data infrastructure is where we start. We are starting with a series of APIs to search, scrape, crawl, map, monitor, and turn the web into clean data for AI
AI might get all the attention, but somebody's gotta keep all that data flowing haha. Love seeing the infrastructure side of the space getting built out too. Nice work guys, congrats on the launch! @hamza_ali59 @arslan_ali34
Macaly
clean web data is the real bottleneck for agents rn 👏 congrats
the monitor piece is the one id want to hear most about. diffing a page is easy, deciding what counts as a change is the actual problem. my first attempt at this fired on every deploy becuase nav markup and timestamps moved, and i had it muted inside a week, which is the worst possible outcome since the alert is still technically working. whats the unit of identity for a monitored page over time? if a site reorganises its url structure or moves the same content behind a different route, does olostep read that as the same page changed or as a page gone. that one decision is what determines whether monitoring survives six months of contact with a real site. and are extraction schemas versioned? when a field stops appearing i want to know whether the site changed or my schema did, and most tools collapse both into null.
Olostep
@rabnoor_s True! The Monitor API is one of the most useful but also technically challenging APIs we have created so far. It puts together most of the foundational building blocks we have built over the years. We have models that first create a graph of the workflow, then we use our own API to take snapshots of the relevant parts of the web when the monitor is created to have a truth baseline. Then we have other in-house models, and harnessing that detect what constitutes a real change worth flagging compared to just noise.