#1
  1. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Feb 2017
    Posts
    68
    Rep Power
    2

    Indexed, though blocked by robots.txt: Need to bother?


    Hi,

    We have intentionally blocked some of the website files which were indexed for years. Now we receive a message "Indexed, though blocked by robots.txt" in GSC. We can ignore as per my knowledge? Are any actions required about this? We thought of blocking them with meta tags but these are PDF files.

    Thanks
  2. #2
  3. Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2013
    Location
    Kolkata,India
    Posts
    72
    Rep Power
    83
    I don't think anything we need to do.The url's were indexed before applying the blocking.You can use the remove url feature from the GSC to remove them from search results.You can also redirect those urls to the other appropriate urls which you can display on Google or in home page.Anyone has other suggestions?
  4. #3
  5. No Profile Picture
    Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Feb 2017
    Posts
    149
    Rep Power
    374
    FYI, just because you block URLs with your Robots.txt file, doesn't mean they can't be indexed - especially if they are linked to from any unblocked pages on your site. If you don't want them to be indexed, you need to add a no-index directive in your Robots.txt file, or add a no-index tag on every single page. And if you don't want them to be crawled at all, you'll also need to add a no-follow tag on every page.
  6. #4
  7. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Aug 2018
    Posts
    26
    Rep Power
    0
    If you closed the pages in robots.txt, I think there's no reason to worry. Give time.

Similar Threads

  1. Sitemap blocked in robots
    By luccwb in forum New User SEO Questions and Answers
    Replies: 3
    Last Post: Aug 5th, 2016, 06:24 AM
  2. URL blocked by Robots.txt Issue in WMT, Pleas Help!!!
    By ansrikanth in forum Search Engine Optimization
    Replies: 2
    Last Post: Nov 15th, 2012, 06:14 AM
  3. Reindexing a site that has been blocked with robots.txt
    By MercJones in forum Google Optimization
    Replies: 0
    Last Post: Jun 8th, 2012, 10:07 AM
  4. Robots.txt allowing everything, Google says 4,200 pages blocked?
    By number1superguy in forum Google Optimization
    Replies: 3
    Last Post: Oct 11th, 2011, 09:39 AM
  5. Google and Noindex blocked by robots.txt
    By roseberry in forum Google Optimization
    Replies: 0
    Last Post: Oct 30th, 2008, 04:26 PM

IMN logo majestic logo threadwatch logo seochat tools logo