Search Technologies
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
 
User Name:
Password:
Remember me
Go Back   SEO Chat Forums > Search Engine Strategies > Search Technologies
Reload this Page

Robots.txt SEO question

Discuss Robots.txt SEO question in the Search Technologies forum on SEO Chat.
Robots.txt SEO question Search Technologies forum discussing search engine algorithm technologies, technical papers, experimental concepts such as fuzzy search and contextual search, and other advances in search.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old February 26th, 2008, 10:34 PM
chilldeals chilldeals is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Mar 2007
Location: AZ
Posts: 12 chilldeals User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 3 h 37 m 47 sec
Reputation Power: 0
Exclamation Robots.txt SEO question
I have affiliate software that clones the destination page with a dynamic url. To avoid duplicate content I noindex /*?* all dynamic pages

If i am tracking a few websites, for monthly sales, who link to this dynamic url (the cloned page the affiliate software created),

and this url is set up for "noindex" in my robots.txt file,

will the few websites linking in still push pagerank through to all the links on this dynamic (cloned) page.
Reply With Quote
  #2  
Old March 2nd, 2008, 12:14 PM
ClickyB's Avatar
ClickyB ClickyB is offline
Scru u guys I'm goin home
SEO Chat Expert (3500 - 3999 posts)
  
Join Date: Mar 2004
Location: Gloucester (South West UK).
Posts: 3,914 ClickyB User rank is Sergeant Major (2000 - 5000 Reputation Level)ClickyB User rank is Sergeant Major (2000 - 5000 Reputation Level)ClickyB User rank is Sergeant Major (2000 - 5000 Reputation Level)ClickyB User rank is Sergeant Major (2000 - 5000 Reputation Level)ClickyB User rank is Sergeant Major (2000 - 5000 Reputation Level)ClickyB User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 3 Months 4 Days 3 h 58 m 36 sec
Reputation Power: 50
Hi cd,

Adding a url to robots.txt doesn't make that page "noindex", it makes it "nocrawl"...
If a url in your robots.txt file has sufficient incoming links it can still be indexed (the url only) but - since the crawlers will avoid spidering the page - none of it's links will receive any "pr juice".
__________________
ClickyB
"The quality of the visitor is more important than the volume".. Egol 22nd Feb 2008
New to SEO? Start Here: SEO FAQ
Forum Rules & Posting Guidelines
Reply With Quote
  #3  
Old March 17th, 2008, 02:30 PM
dzine's Avatar
dzine dzine is offline
Vergruizer: Vot tebe khuy
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Oct 2005
Location: in a life preserver @ seorefugee
Posts: 1,918 dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Month 5 Days 19 h 59 m 20 sec
Reputation Power: 23
Yeah, use the <meta name="robots" content="noindex"> tag. It will take a while....

There are shortcuts but they're risky.
Reply With Quote
  #4  
Old March 17th, 2008, 03:52 PM
Jean-Luc Jean-Luc is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Dec 2004
Location: Brussels, Belgium
Posts: 392 Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 21 m 48 sec
Reputation Power: 7
Quote:
Originally Posted by chilldeals
I have affiliate software that clones the destination page with a dynamic url. To avoid duplicate content I noindex /*?* all dynamic pages
Iwould prefer to use a 301 redirect from these affiliate links to the official destination URL. This would concentrate the link juice on the official destination URL. IMO, that's better than to waste it with noindex or robots.txt.

Jean-Luc
__________________
AWStats Support : add-on's, extra sections, forum, installation assistance
AWStats remote service for less than $2 a month
Checking redirects is now as easy as 1 2 3, even if you are not a HTTP-header guru !
Reply With Quote
  #5  
Old May 15th, 2008, 10:44 PM
nasskobar nasskobar is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Mar 2008
Posts: 10 nasskobar User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 4 h 7 sec
Reputation Power: 0
Question Could you clarify
Quote:
Originally Posted by Jean-Luc
Iwould prefer to use a 301 redirect from these affiliate links to the official destination URL. This would concentrate the link juice on the official destination URL. IMO, that's better than to waste it with noindex or robots.txt.

Jean-Luc


Could you explain that a bit more clearly please? I have not understood what do you mean by "official destination URL".

Do you mean you would use a 301 redirect code to redirect all the affiliate links to your website to a single URL? Is that correct?

Please clarify.

Thank you.
Reply With Quote
  #6  
Old May 20th, 2008, 01:03 AM
Jean-Luc Jean-Luc is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Dec 2004
Location: Brussels, Belgium
Posts: 392 Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 21 m 48 sec
Reputation Power: 7
Quote:
Originally Posted by nasskobar
Do you mean you would use a 301 redirect code to redirect all the affiliate links to your website to a single URL?
That's correct.

With a 301 redirect from all affiliate links to a unique "destination URL", you will concentrate the link juice on the "destination URL".

Jean-Luc
Reply With Quote
  #7  
Old May 20th, 2008, 01:11 AM
nasskobar nasskobar is offline
Registered User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Mar 2008
Posts: 10 nasskobar User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 4 h 7 sec
Reputation Power: 0
Thumbs up Thank you
Quote:
Originally Posted by Jean-Luc
That's correct.

With a 301 redirect from all affiliate links to a unique "destination URL", you will concentrate the link juice on the "destination URL".

Jean-Luc


Thank you for the clarification Jean!
Reply With Quote
  #8  
Old May 20th, 2008, 01:53 AM
full house's Avatar
full house full house is offline
Contributing User
SEO Chat Novice (500 - 999 posts)
  
Join Date: Apr 2007
Location: Philippines
Posts: 659 full house User rank is Corporal (100 - 500 Reputation Level)full house User rank is Corporal (100 - 500 Reputation Level)full house User rank is Corporal (100 - 500 Reputation Level)full house User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 3 Days 5 h 8 m 47 sec
Number of bans: 1
Reputation Power: 4
Send a message via Yahoo to full house Send a message via Google Talk to full house
Does 301 redirect help not to be index by google? based on chilldeals question, and he wants to avoid duplicate content.

I think <meta name="robots" content="noindex"> could help.
Reply With Quote
Reply

Viewing: SEO Chat Forums > Search Engine Strategies > Search Technologies > Robots.txt SEO question

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 6 hosted by Hostway.