Sitebulb is a multi-award winning desktop website crawler that delivers enterprise level analysis.
Sitebulb is an extremely powerful website crawler with the world's most insightful reporting system, winner of 'Best Search Software Tool' at the 2018 UK Search Awards and the US Search Awards.
No reviews yetBe the first to leave a review for Sitebulb
I've been using the beta of this product and it's fantastic. Has replaced Deep Crawl, the former gold standard in technical SEO crawl tools, and gives a nice clear overview of issues to fix. I'm sure there's more to come and they've been responsive to feedback and bugs through the beta, so looking forward to seeing how Sitebulb continues to improve in future.
Question for you @hathawayp; where do you see Sitebulb going in the future? Any features you were keen to include that didn't make it to this initial launch?
@tomdavenport Hey Tom, thanks for the kind words :)
There are a few big features which we are really keen to build, which definitely seem to be in demand based on our feedback so far:
- 'Printable' Reports: Generating PDF reports automatically for users, either to be shared internally or sent to clients.
- Segments: Allowing users to segment their site based on page type (e.g. 'products', 'blog posts') and building reports/drilldowns based on these segments.
- Scheduling: Allowing users not only to queue (as they can now) but also to schedule crawls to run at a specific time.
Report
Looks great, guys. Quick question - realistically, on a site with say 100,000 URLs (so not absolutely huge) how long would you expect a complete crawl to take?
@staceycav Hey Stacey. Well, that question is a little bit like 'how long is a piece of string'. There are a number of variables that affect how long a crawl will take, the biggest one obviously being how many URLs there are. And a site with 100,000 URLs indexed in Google could easily have 300,000 URLs when you include resources, external links etc...
But for the sake of argument let's call it 100,000 HTML URLs in total, with no page resources switched on. On most websites it's not healthy (not to mention polite) to crawl TOO fast, so we recommend a speed of 5 URLs/second. At that rate you can crawl a 100k site in about 6 hours (including time to build the reports at the end).
However, sometimes even 5 URLs/second is too fast for some websites, and can cause the server to slowdown. Sitebulb is able to detect this and slow it's crawling rate down, so as not to cause any user disruption. For example, I crawled a 100k page site the other day and it took almost exactly 12 hours (~2.8 URLs/second).
I've written more on all this here:
https://sitebulb.com/resources/g...
Thanks for the question!
Report
I've worked with Botify, screaming frog, the link-assistant tool, seomator, raven, moz and others. I think all of them? This is very much the audit tool that I've been looking for.
Pros:
Beautifully presented actionable data. Clear. Easy to use. Fast. No limits to usage. Low system resource guzzling.
Cons:
One or two very minor reporting issues. Fully expect these to be fixed soon.
My favorite part of Sitebulb, especially compared to other, similar products has been the organization of the crawl information. It's just a little bit more in-line with what I'm looking for when auditing a website for any issues.
@fighto Thanks for the comment Paul. We tried as best we could to approach it from the perspective of: 'when doing an audit for SEO, how do users work through each section.'
Although we had to rip it apart and put it back together again more than a few times!
Report
I've been fortunate enough to be able to test out @Sitebulb as part of the beta release and I am also a long term user of @urlprofiler (also by Gareth and Patrick)
This is a great tool and I have had very few issues crawling sites in excess of 100k URLs
Pros:
Crawl Map Visualisations, Easy to understand reports
My agency has been a beta user for the last 6 months and we're pleased with where the product is at.
It's better than certain loud amphibians and tech crawlers named after evil space aliens.
Sitebulb
Sitebulb
I've worked with Botify, screaming frog, the link-assistant tool, seomator, raven, moz and others. I think all of them? This is very much the audit tool that I've been looking for.
Pros:Beautifully presented actionable data. Clear. Easy to use. Fast. No limits to usage. Low system resource guzzling.
Cons:One or two very minor reporting issues. Fully expect these to be fixed soon.
Sitebulb
Dropit for Clubhouse
Sitebulb
I've been fortunate enough to be able to test out @Sitebulb as part of the beta release and I am also a long term user of @urlprofiler (also by Gareth and Patrick)
This is a great tool and I have had very few issues crawling sites in excess of 100k URLs
Pros:Crawl Map Visualisations, Easy to understand reports
Cons:White label Reporting for Agencies
Sitebulb
Sitebulb
Sitebulb
Intellifluence
Sitebulb