Google Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
Go Back   SEO Chat ForumsGoogleGoogle Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old March 5th, 2008, 12:13 PM
CommonDavid CommonDavid is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Dec 2007
Location: Manchester NH
Posts: 365 CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 4 Days 4 h 7 m 52 sec
Reputation Power: 6
Send a message via ICQ to CommonDavid Send a message via AIM to CommonDavid
Difference btw rel="nofollow" & putting urls in robots.txt

I've been going through all my sites and using nofollow's all over the place, but I've been wondering lately what the difference is between using the rel="nofollow" and putting the url's in the robots.txt file?

If the url is in the robots.txt file will it still receive PR juice?

Reply With Quote
  #2  
Old March 5th, 2008, 12:20 PM
Swirleigh's Avatar
Swirleigh Swirleigh is offline
SEO Gawdess
SEO Chat Novice (500 - 999 posts)
 
Join Date: Jul 2006
Location: San Francisco Bay Area
Posts: 586 Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 9 h 45 m 59 sec
Reputation Power: 10
Blocking the urls from indexing in your robots.txt prevents crawlers from visiting them at all, or listing them.

Ref=nofollow passes no link juice, but the urls may get followed and indexed anyway. Yahoo follows them.

Danny Sulivan says "Google WILL follow nofollow links in the sense that if someone else links to a page without using nofollow, then Google might find the page that way. Nofollow on one link to a page doesn't somehow cause ALL links to be off limits. "
Comments on this post
CommonDavid agrees: Perfect! Thanks for the excellent response.

Reply With Quote
  #3  
Old March 5th, 2008, 12:35 PM
CommonDavid CommonDavid is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Dec 2007
Location: Manchester NH
Posts: 365 CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 4 Days 4 h 7 m 52 sec
Reputation Power: 6
Send a message via ICQ to CommonDavid Send a message via AIM to CommonDavid
Quote:
Originally Posted by Swirleigh
Blocking the urls from indexing in your robots.txt prevents crawlers from visiting them at all, or listing them.

Ref=nofollow passes no link juice, but the urls may get followed and indexed anyway. Yahoo follows them.

Danny Sulivan says "Google WILL follow nofollow links in the sense that if someone else links to a page without using nofollow, then Google might find the page that way. Nofollow on one link to a page doesn't somehow cause ALL links to be off limits. "


I was just thinking about this, and it's an important thing to use a nofollow, but not to block the url with the robots.txt if you've got pertitiant info on the page, and/or if you've got sitelinks or plan on getting them,

ex: you might not want to waste your PR on the "Driving Directions" page, but you still want to see that sitelink show up in your listing ;-)

Thanks again

Reply With Quote
  #4  
Old March 5th, 2008, 02:51 PM
Sniffer1 Sniffer1 is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: May 2005
Posts: 76 Sniffer1 User rank is Lance Corporal (50 - 100 Reputation Level)Sniffer1 User rank is Lance Corporal (50 - 100 Reputation Level)Sniffer1 User rank is Lance Corporal (50 - 100 Reputation Level) 
Time spent in forums: 1 Day 23 m 21 sec
Reputation Power: 5
Quote:
Originally Posted by CommonDavid
I was just thinking about this, and it's an important thing to use a nofollow, but not to block the url with the robots.txt if you've got pertitiant info on the page, and/or if you've got sitelinks or plan on getting them,

ex: you might not want to waste your PR on the "Driving Directions" page, but you still want to see that sitelink show up in your listing ;-)

Thanks again


how do you block google following an outgoing link in your robots file?

Can someone show me the code?

Last edited by Sniffer1 : March 5th, 2008 at 02:53 PM.

Reply With Quote
  #5  
Old March 5th, 2008, 03:15 PM
Swirleigh's Avatar
Swirleigh Swirleigh is offline
SEO Gawdess
SEO Chat Novice (500 - 999 posts)
 
Join Date: Jul 2006
Location: San Francisco Bay Area
Posts: 586 Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 9 h 45 m 59 sec
Reputation Power: 10
No way to disallow indexing or following of an outbound link in your robots.txt file. Your robots.txt list files or sections of your site you want the robots/search engine crawlers to stay out of and not index.

User-agent: *
Disallow: /real-estate-articles/


Wow, wouldn't that be great if you could disallow outgoing links?! List all your competitors links and disallow indexing of them. heehee

Reply With Quote
  #6  
Old March 5th, 2008, 03:56 PM
CommonDavid CommonDavid is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Dec 2007
Location: Manchester NH
Posts: 365 CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level)CommonDavid User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 4 Days 4 h 7 m 52 sec
Reputation Power: 6
Send a message via ICQ to CommonDavid Send a message via AIM to CommonDavid
The robots.txt is really just a way to make it so the spider doesn't index admin pages, and in one of my personal instances: pages that let a user add a review. But the more I think about it, the more I realize that that's all you should really put in there, and then just use the nofollow for everything else, the way that seochat has done here.

Reply With Quote
  #7  
Old March 5th, 2008, 04:07 PM
Swirleigh's Avatar
Swirleigh Swirleigh is offline
SEO Gawdess
SEO Chat Novice (500 - 999 posts)
 
Join Date: Jul 2006
Location: San Francisco Bay Area
Posts: 586 Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level)Swirleigh User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 9 h 45 m 59 sec
Reputation Power: 10
Not just admin pages. How about member's only areas? What good is it to charge a fee, or to only display content after user becomes a lead, if they can view the same content viewing Google's cache? There are many area's you might want to disallow indexing for.

cgi bin
drafts
private
duplicate printable versions
iframes content, that have no navigation
members only

Reply With Quote
Reply

Viewing: SEO Chat ForumsGoogleGoogle Optimization > Difference btw rel="nofollow" & putting urls in robots.txt


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump



 Free IT White Papers!
 
How to Present Effectively Online
This white paper offers practical and actionable advice on the key steps that any presenter should consider as they plan and execute a Webinar or online meeting.

Request Your Free Technology Downloads!
 
Open Source Security Myths
Open Source Software (OSS) is computer software whose source code is available to the general public with relaxed or non-existent intellectual property restrictions (or arrangement such as the public domain), and is usually developed with the input of many contributors.

Request Your Free Technology Downloads!
 
Power and Cooling Capacity Management for Data Centers
This paper describes the principles for achieving power and cooling capacity management.

Request Your Free Technology Downloads!
 
Scalable, Fault-Tolerant NAS for Oracle - The Next Generation
For several years NAS has been evolving as a storage alternative for Oracle databases, and for good reason: NAS is quite often the simplest, most cost-effective storage approach for Oracle. Learn about the benefits that HP's approach to scalable NAS brings to Oracle environments in this comprehensive white paper.

Request Your Free Technology Downloads!
 
Understanding Web Application Security Challenges
This white paper discusses many common threats and preventive measures for Web application security, and explains what you can do to help protect your organization.

Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2009 by Developer Shed. All rights reserved. DS Cluster 4 hosted by Hostway
Stay green...Green IT