1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Oct 2013
    Rep Power

    Google Indexation Issue

    We had our search pages being indexed by Google up until a few months ago. We first added the noindex,nofollow tag to the search results, but soon added the /search.php to the robots.txt. Since then our Index status has risen more than 20,000 in GWT to over 100,000. Doing a site: search also gives over 140,000 pages. If I do site:example inurl:search it shows almost 50,000 results.

    Did I jump the gun with adding search to the robots.txt which just made the pages uncrawlable but still in Googles index? Should I take search out of the robots.txt and give Google more time to noindex those pages?
  2. #2
  3. Philip@SearchBenefit.com
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Oct 2009
    Massachusetts, USA
    Rep Power
    Let me make sure I fully understand what you are saying. You added search.php to results on what mode, allow or disallow? Can you post your robots.txt or at least the relevant part?

    What you may be seeing are old cached versions of the pages. Check the page source code of the cached pages to see if noindex is on them.

    In general, you may well want a selection of your searches indexed, but you don't want a lot of random and possibly very similar search result pages spamming the index. I would create static category pages and populate them with the relevant search results, for example, while disallowing search.php.

Similar Threads

  1. Google Crawling and Indexation 101
    By PhilipSEO in forum Google Optimization
    Replies: 26
    Last Post: Feb 2nd, 2015, 03:52 AM
  2. Google (good) News - bad indexation?
    By sambkk in forum Google Optimization
    Replies: 7
    Last Post: May 29th, 2012, 12:39 AM
  3. Problems with Google indexation
    By HunterThompson in forum Google Optimization
    Replies: 1
    Last Post: Aug 13th, 2006, 06:02 AM
  4. google indexation speed
    By seochat in forum Google Optimization
    Replies: 4
    Last Post: May 23rd, 2005, 03:04 AM

IMN logo majestic logo threadwatch logo seochat tools logo