1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Nov 2016
    Rep Power

    To what extend does page-speed affect crawler?

    I'm curious since my website is not always that fast... ;-)

    Does a crawler such as google offer a day/week time slot? Or is there a num pages slot?

    In the first case the page speed heavily affects the number of pages/updates getting indexes while obviously when there is a fixed #pages slot it does not do much.

  2. #2
  3. Dinosaur
    SEO Chat Super Genius (4500 - 4999 posts)

    Join Date
    Jun 2011
    Rep Power

    I have shared the above image a few times now.
    But the big spikes and the big dips are directly related to the site speed at the time. As you can see when the site is fast google crawl a lot more pages than when its slow so yes it appears google set a maximum of time per site for gathering data.
    Owner of Page Explorer the page onsite SEO checker
    Useful Tools: Site Crawler: Screaming Frog | Free SSL: Cloudflare | Backlinking 101: Backlinks in 2018
  4. #3
  5. No Profile Picture
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Mar 2017
    Sydney, Australia
    Rep Power
    As I understand it, the basic answer is, it depends...

    Googlebot "learns" how frequently your site pages are changed and automates its re-visit cycle, partly around your page change history.

    It is also possibly finding your site via external links. That means it could be hitting your site on any page and starting its crawl from that point. Your volume/frequency of external links updates could impact on this attribute.

    There is also a crawl assessment factor that looks at the demand on your hosting server. If you are on a hosting service that crams up its ability to process access demands, Googlebot will back off if it thinks it is creating a drain on your hosting server's processing power.

    These factors are not so much load speed factors as you can measure with the online Google tool.

    How important is it for your site to be re-indexed quickly?

    How big is your site? If it is less than 1,000 pages, your concern may be academic.

Similar Threads

  1. Does Loading Speed Affect Ranking?
    By GraceLoveSEO in forum Search Engine Optimization
    Replies: 48
    Last Post: Apr 26th, 2015, 06:41 PM
  2. Google Webmaster tools (Crawler speed Question)
    By lovekills_s in forum Google Optimization
    Replies: 5
    Last Post: Apr 11th, 2007, 04:03 AM
  3. Server speed have an affect on google rankings?
    By mightywren in forum Google Optimization
    Replies: 11
    Last Post: Oct 4th, 2006, 08:58 AM
  4. Server speed have an affect on site rankings?
    By mightywren in forum Search Engine Optimization
    Replies: 2
    Last Post: Oct 3rd, 2006, 08:40 PM
  5. Does Google's Toolbar Affect Connection Speed?
    By iceglobe in forum Google Optimization
    Replies: 1
    Last Post: Mar 7th, 2003, 04:00 PM

IMN logo majestic logo threadwatch logo seochat tools logo