#1
  1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Jun 2004
    Posts
    12
    Rep Power
    0

    can u block robots accessing https pages with robots.txt?


    Hello All,

    Ok. I have some pages that google has cached and they are getting the december 31 1969 cache and i think its because of duplicate content. Google has cached a https page and a http version of the same page. I used to have the page in secure mode but then changed it. Now, is there anyway to disallow caching of my pages in https? I read the google robots.txt information and they say that you should have a seperate robots.txt for http and https. How would you do this? Since it is a .txt file there is no way to run a coldfusion script to check the port and then serve up content based upon the port 80 or 443. Should I not even be trying to do this is it not a big deal? I just want to fix the pages so that google will start caching them again. Thanks a lot!
  2. #2
  3. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Jun 2004
    Posts
    12
    Rep Power
    0
    I was also wondering if maybe this can be achieved using absolute urls? Can u use absolute urls?
  4. #3
  5. http://tinyurl.com/cz56g
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Sep 2004
    Location
    D0RDRECHT NL
    Posts
    6,063
    Rep Power
    31
    No, unless your https files are in a separate directory or have another distinguishing filename prefix (example: somepage.html vs. sec-somepage.html) then you can't use robots.txt for that...

    If somehow you can get the https versions to automatically include the robots META tag like so: <meta name="robots" content="noindex">, then that might help (in time)...
    ...please help me w/ the real Redscowl Bluesingsky...how2 check backlinks...now postin' @ SEO Refugee ...
    <`)~ LOL now that I finally have a paypal account, I'm charging 19,- for SEO advice via PM. Seriously...
  6. #4
  7. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Jun 2004
    Posts
    12
    Rep Power
    0
    Thats a good idea! It will be no problem at all I just gotta put a little piece of code in my application.cfm file! THANK U!!!!!!
  8. #5
  9. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Feb 2009
    Posts
    2
    Rep Power
    0
    I dont know if this would work (I know this is old)

    <script language="javascript">
    if (window.location.protocol != "https:") {
    document.write('<meta name=robots content=noindex>');
    }
    </script>

    Comments on this post

    • dzine : (0) No it would not work :eek:
  10. #6
  11. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    May 2008
    Location
    Ontario, Canada
    Posts
    223
    Rep Power
    17
    Try using the new canonical tag supported by Google/Yahoo/MSN.

    http://forums.seochat.com/google-news-46/google-and-yahoo-release-new-duplicate-content-tag-236456.html
  12. #7
  13. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Feb 2009
    Posts
    2
    Rep Power
    0
    Ah! Of course! Silly... Sometimes I try to do things the hard way... I just read about that a week ago as well. Thanks so much!
  14. #8
  15. Super Moderator
    SEO Chat Genius (4000 - 4499 posts)

    Join Date
    Apr 2006
    Location
    I N D I A
    Posts
    4,141
    Rep Power
    1028
    Guys from where did u pull out this 4+ yrs old corpse?

    Please let it rest in peace.

    Thread closed!

    SEO FAQs - You might find your answer here.
    SEOchat Forum Rules - Read Before You Post

Similar Threads

  1. How to handle URL changes of pages?
    By dan in forum Google Optimization
    Replies: 9
    Last Post: Feb 10th, 2003, 02:35 AM

IMN logo majestic logo threadwatch logo seochat tools logo