#1
  1. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Feb 2015
    Location
    Warrington, UK
    Posts
    138
    Rep Power
    222

    Robots.txt Disallow Issue


    Hi Guys,

    Can anyone explain why if I do a site:openhouseestateagents.net query in Google, the pages are indexed, despite the robots.txt file being disallowed

    openhouseestateagents.net/robots.txt

    Thanks,

    Will
    Last edited by dzine; Mar 19th, 2016 at 11:31 AM. Reason: Please make sure you de-link your URLs
  2. #2
  3. SEO Insultant
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Oct 2005
    Location
    sharing a room with my ego
    Posts
    6,267
    Rep Power
    2846
    When did you add the robots.txt file? Before or after the pages were indexed?
  4. #3
  5. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2016
    Location
    India
    Posts
    14
    Rep Power
    0
    If you have allowed robot.txt it will be indexed , you must have disallowed robot.txt

    I have same question what dzine asked you "When did you add the robot.txt file? Before or after the pages indexed?"

    Did your problem solved?

    If yes then how?

    Please answer.

Similar Threads

  1. Robots Disallow: /*?
    By xeont in forum Google Optimization
    Replies: 3
    Last Post: Apr 26th, 2012, 02:50 PM
  2. Robots.txt DisAllow
    By rizmriz in forum Google Optimization
    Replies: 4
    Last Post: Apr 18th, 2012, 11:47 AM
  3. Some one used disallow / on my robots.txt
    By beanbaby in forum Google Optimization
    Replies: 25
    Last Post: Jan 5th, 2007, 05:04 AM
  4. Robots.txt... should I disallow....
    By seofishin in forum Google Optimization
    Replies: 6
    Last Post: Aug 16th, 2005, 11:50 AM
  5. robots.txt disallow
    By mario in forum Google Optimization
    Replies: 9
    Last Post: May 20th, 2003, 05:29 PM

IMN logo majestic logo threadwatch logo seochat tools logo