#1
  1. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2015
    Location
    Pinkcity
    Posts
    59
    Rep Power
    106

    In robots.txt file avoid duplicacy of new landing page content


    Hello,
    I need a help in creating new adwords landing pages. Current landing page in adwords is same as it is (ex: domain.com/product1.aspx), but for experiment, I have designed new landing page(ex: domain.com/ppc/product1.aspx) for adwords only. Content would be exactly same, the only changes is its design.
    Now I am disallowing this folder(/ppc/) using robots.txt file(user-agent: *) to avoid duplicacy. Do I need to allow google Adsbot in the file?

    In robots.txt file,

    Code:
    User-Agent: *
    Disallow: /ppc/
    would be enough or I have to add

    Code:
    User-Agent: Adsbot-Google
    Allow: /ppc/
  2. #2
  3. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Oct 2009
    Location
    Detroit, MI USA
    Posts
    178
    Rep Power
    557
    Originally Posted by uditsh
    Hello,
    I need a help in creating new adwords landing pages. Current landing page in adwords is same as it is (ex: domain.com/product1.aspx), but for experiment, I have designed new landing page(ex: domain.com/ppc/product1.aspx) for adwords only. Content would be exactly same, the only changes is its design.
    Now I am disallowing this folder(/ppc/) using robots.txt file(user-agent: *) to avoid duplicacy. Do I need to allow google Adsbot in the file?

    In robots.txt file,

    Code:
    User-Agent: *
    Disallow: /ppc/
    would be enough or I have to add

    Code:
    User-Agent: Adsbot-Google
    Allow: /ppc/
    Using the robots.txt would be a good way to prevent spiders from crawling that content.

    I guess some others on here can answer better than me, but i wonder if using a canonical tag ono the PPC page would work as well. I have not test this so i'll let someone like @chedders answer this. I guess my thinking with going with the canonical tag would be that you can still track the ppc/product.aspx page using GA
  4. #3
  5. Super Moderator
    SEO Chat Genius (4000 - 4499 posts)

    Join Date
    Apr 2006
    Location
    I N D I A
    Posts
    4,141
    Rep Power
    1026
    This is what you need

    # Blocking all spiders from accessing the PPC folder
    user-agent: *
    disallow: /ppc/

    # Allowing only Google Adsbot to access the website
    user-agent: AdsBot-Google
    disallow:

    Comments on this post

    • uditsh agrees

    SEO FAQs - You might find your answer here.
    SEOchat Forum Rules - Read Before You Post
  6. #4
  7. SeoRaptor
    SEO Chat Adventurer (500 - 999 posts)

    Join Date
    Mar 2016
    Location
    France, Saint-Etienne
    Posts
    997
    Rep Power
    1703
    You could add a noindex tag to your landing page header, this way you would be sure Google won't index it.

    Restricting access to bot using robots.txt is not the best option for two reasons, firstly Google bot doesn't always follow you robots.txt directives, secondly if anyone links to your landing page on the web it might be scrapped and indexed.
    Owner of Bennetonable - "My opinions are my own - Feel free to disagree & think above the fold."
    Need a quick analysis > Free SEO grader Beta

Similar Threads

  1. Duplicacy of Content Issue In E-commerce sites
    By sarthakgupta101 in forum Search Engine Optimization
    Replies: 0
    Last Post: Jul 17th, 2013, 02:22 PM
  2. Hiding page content from robots
    By davidb2002 in forum Search Engine Optimization
    Replies: 6
    Last Post: Nov 6th, 2008, 10:31 PM
  3. Top Content Page NOT in Top Landing Pages
    By e-milosz in forum Google Optimization
    Replies: 1
    Last Post: Aug 2nd, 2008, 02:17 PM
  4. Can robots avoid part of a page
    By ppp in forum Google Optimization
    Replies: 4
    Last Post: Jun 17th, 2008, 03:44 AM

IMN logo majestic logo threadwatch logo seochat tools logo