#1
  1. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2012
    Posts
    31
    Rep Power
    7

    Can google access this area of my server?


    I have my main website at httpdocs/ i have another site i am working on which is installed on httpdocs/dirname/ . google crawls my main website but could they also be able to crawl my httpdocs/dirname/ folder too? I have no links from the main site directing to this sub dir folder.
  2. #2
  3. Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2014
    Location
    India
    Posts
    43
    Rep Power
    28
    Google can able to crawl all your sub folders unless it is restricted by .htaccess/.robots.txt/URL forwarding.
  4. #3
  5. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2012
    Posts
    31
    Rep Power
    7
    Originally Posted by sathishvaiha
    Google can able to crawl all your sub folders unless it is restricted by .htaccess/.robots.txt/URL forwarding.

    So for robot text file i would put the following to not allow it to crawl this sub dir

    Disallow: /subdir

    OR

    Disallow: /subdir/

    which one?
  6. #4
  7. Super Moderator
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Mar 2004
    Location
    Gloucester (South West UK).
    Posts
    6,533
    Rep Power
    3521
    Disallow: /subdir
    ClickyB
    "The quality of the visitor is more important than the volume..." (Egol 22nd Feb 2008)
    [New to SEO/SeoChat?] [Canonical Problems?] [Forum Rules & Posting Guidelines]

Similar Threads

  1. US Server vs. Foreign Server on Google
    By michaelswoops in forum Google Optimization
    Replies: 1
    Last Post: Nov 30th, 2010, 10:52 PM
  2. Area of my site now doing badly in google
    By Glenn Kilpatric in forum Google Optimization
    Replies: 7
    Last Post: Oct 18th, 2008, 03:06 PM
  3. Truckers sue over access to Los Angeles-area ports (AP)
    By RSS_News_User in forum Science News
    Replies: 0
    Last Post: Jul 29th, 2008, 06:04 AM
  4. Replies: 0
    Last Post: Feb 23rd, 2006, 07:02 PM
  5. Replies: 0
    Last Post: Feb 23rd, 2006, 06:02 PM

IMN logo majestic logo threadwatch logo seochat tools logo