#1
  1. No Profile Picture
    Newbie
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2017
    Posts
    4
    Rep Power
    0

    SEO and Robots.txt related question


    Hey everyone,

    I'm new here, I had a question and I was hoping you could perhaps help me out with it.

    It's related to SEO and the use of a Robots.txt file.

    I have an index.html on my domain as an entrance page that links to the main page, say in a /main folder. I have every page EXCEPT
    for the domain root index.html file blocked by robots so they do not get indexed. Only the entrance page is indexed.

    Will this negatively affect SEO?? Will Google (search engines) only read my entrance page and think my website is a 1 page site
    or will it scan the entire website and know that it's a massive website but it will only index the entrance page like I want.??

    Also, I use Google analytics so I'm not sure if they are tied together but through analytics Google should know that it is a large
    website and people are using it, and also get scans of everything needed for knowing if a website has important content.... am I right?

    Anyone know about this sort of thing? I've been a webmaster for 4 years but I really don't know all that much about SEO, I think I
    got lucky for a few years but now I need to learn some, lol.

    Any help would be great thanks
  2. #2
  3. Dinosaur
    SEO Chat Scholar (3000 - 3499 posts)

    Join Date
    Jun 2011
    Location
    UK
    Posts
    3,380
    Rep Power
    6107
    No indexing your site except for one page in googles eyes they are left with 1 page in the index so yes it will be treated single page website.
    I think your assuming that as google bot crawls your site its also doing the indexing. I am afraid google bot just gathers data / pages and the results of that crawl gets passed to a different system for indexing. Your pages wont be reaching that system

    Comments on this post

    • Will-O-The-Wisp agrees
    Owner of Page Explorer the page onsite SEO checker
    Useful Tools: Site Statistics: SEM Rush | Site Crawler: Screaming Frog
  4. #3
  5. No Profile Picture
    Newbie
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2017
    Posts
    4
    Rep Power
    0
    That's interesting I never knew that. Thanks for your reply.

    So I shouldn't use a robots.txt to block the rest of my site if I only want the entrance page to be indexed? Google will somehow know to just index my entrance page and not the rest of the website?

    Thanks again,

  6. #4
  7. Dinosaur
    SEO Chat Scholar (3000 - 3499 posts)

    Join Date
    Jun 2011
    Location
    UK
    Posts
    3,380
    Rep Power
    6107
    If you dont want the rest of your site in the index then robots.txt is the correct way to go.
    however having a single page website is notoriously difficult to rank because on its own it will have little meaning nor will it gain any page rank from the rest of your site as they have been dropped from the index so even if they have backlinks the page rank is effectively killed.

    I am struggling with why you would build a site that is publically available and dont want indexed, I can fully understand if its your admin section but that is normally hidden behind some form of security so google has no access anyway. Or maybe a members only area but again security will take care of that also.

    Normally I would suggest users leave their robots.txt well alone and let google do its thing, its only if things are going wrong that you should start messing with this file.

    If your trying to do page rank sculpting this is not the way to do that and will end up only remove page rank / link juice from your site so creating a weaker version.
    Owner of Page Explorer the page onsite SEO checker
    Useful Tools: Site Statistics: SEM Rush | Site Crawler: Screaming Frog
  8. #5
  9. No Profile Picture
    Newbie
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2017
    Posts
    4
    Rep Power
    0
    That's for the reply again.

    The main reason I want to have the entire site hidden and only the entrance page indexed is because it is an 18+ adults only website. I don't people coming directly onto a page without first agreeing to a disclaimer / age warning page. In the past I have had just the main index landing page hidden and just the entrance page but over the years things have changed and I even changed my websites. They are now based on WordPress and because of this I can allow search engines to index all pages because I have a plugin that pops up no matter what page people come in through with the warning and it makes a cookie.

    Recently I have started downgrading my websites and removing features and services. I had improved them all too much and people weren't buying anything anymore, since downgrading and removing features sales are back up. I am thinking an even better downgrade for my website would be to get rid of the WordPress entirely and start over just with html5 static pages, it should increase sales even further. Problem is if I do that I don't have the plugin / cookie with age warning / disclaimer popping back up again.

    I suppose it may be possible to build a JavaScript thing and make a cookie after serving visitors an age warning / disclaimer but I don't know JavaScript.

    This is the main reason I don't want the website all indexed, just the entrance page.
  10. #6
  11. Dinosaur
    SEO Chat Scholar (3000 - 3499 posts)

    Join Date
    Jun 2011
    Location
    UK
    Posts
    3,380
    Rep Power
    6107
    Your requirement is quite small, if you cant code the function yourself you could nip over to fiverr and hire someone
    Owner of Page Explorer the page onsite SEO checker
    Useful Tools: Site Statistics: SEM Rush | Site Crawler: Screaming Frog

Similar Threads

  1. Related Question
    By vrvertex in forum Search Engine Optimization
    Replies: 2
    Last Post: Jul 10th, 2008, 06:23 AM
  2. I have one Keyword related Question
    By beamup in forum Keyword Research
    Replies: 7
    Last Post: Oct 4th, 2007, 01:15 AM
  3. Question related to subdomain
    By socialguy in forum Google Optimization
    Replies: 2
    Last Post: Aug 29th, 2007, 11:56 PM
  4. Replies: 4
    Last Post: Feb 10th, 2006, 12:22 PM

IMN logo majestic logo threadwatch logo seochat tools logo