1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Aug 2007
    New Plymouth, New Zealand
    Rep Power

    How to reduce googlebot's traffic without harming ranking?

    I run a New-Zealand-specific Drupal 7 site for an NPO which is currently exceeding International Traffic quota on our server. While most of the excess is due to spambots hitting the registration page (in the process of dealing with that) I noticed that googlebot is massing up 1.2GB/month in crawling. (Most other bots are under 100MB)
    I'm not sure how many pages we have but it's growing steadily. I'm also not sure how often we are being crawled (PR is 4).

    Is there any way to reduce the amount of traffic from googlebot without harming ranking too much? Would a sitemap help?
  2. #2
  3. Digital Marketing
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Aug 2008
    Rep Power
    can you reduce the frequency of crawls in GWT?
  4. #3
  5. No Profile Picture
    Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Aug 2013
    Rep Power
    In an XML Sitemap, there are tags to specify the Frequency of change ( <changefreq> )and the Priority for crawling ( <priority> )on a page by page level. These are optional tags, but Google should honor them if implemented correctly.

    The <changefreq> tag can be set from Never to Always, with popular frequencies also being Yearly, Monthly, Weekly, Daily.

    The <priority> tag ranges from O to 1, with decimal intervals (.1, .2, .3, etc) in between. The default for a homepage is 1 and all other pages are .5, unless otherwise specified.

    If you choose to modify the XML Sitemap, I would gauge the type of content on the site and determine the minimal crawling needs for the site to keep the search engines aware of any fresh content and throttle the crawling frequency to just above that and set priorities based on the importance of the page and it's need to be crawled relative to the other pages. Refer to sitemaps dot org for more information.

    Hope that helps!

    Comments on this post

    • SidSilhouette agrees

Similar Threads

  1. Do ads reduce Bing ranking?
    By timtak in forum BING/Yahoo Search Optimization
    Replies: 8
    Last Post: Nov 14th, 2012, 07:06 PM
  2. Reduce in Traffic on website
    By ronitmathurseo in forum Search Engine Optimization
    Replies: 10
    Last Post: Sep 24th, 2012, 08:56 AM
  3. How to reduce bounce rate and increase more traffic on the blog ??
    By adleyduke in forum Google Optimization
    Replies: 9
    Last Post: Sep 27th, 2011, 05:14 AM
  4. Hardly any traffic from googlebot
    By earlpearl in forum Google Optimization
    Replies: 34
    Last Post: Jun 16th, 2004, 11:10 AM

IMN logo majestic logo threadwatch logo seochat tools logo