Search Engine Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
 
User Name:
Password:
Remember me
Go Back   SEO Chat ForumsSearch Engine StrategiesSearch Engine Optimization

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old May 1st, 2003, 10:55 PM
brendabee's Avatar
brendabee brendabee is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Posts: 78 brendabee User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 6
NO FOLLOW

I am trying to keep SE bots out of some of my pages. What's the best way to do this? I have tried using
<META NAME="robots" content="noindex,nofollow">

Freshbot came through one of my sites and completely ignored it.

I've read that Googlebot ignores meta tags. Is there another way? I'd like to avoid putting it in the robots.txt file.

Reply With Quote
  #2  
Old May 1st, 2003, 10:59 PM
kapsat's Avatar
kapsat kapsat is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Location: USA
Posts: 150 kapsat User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 4 m 54 sec
Reputation Power: 6
why do you not want to put a robots.txt file

kapsat
__________________
kapsat

Reply With Quote
  #3  
Old May 1st, 2003, 11:07 PM
brendabee's Avatar
brendabee brendabee is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Posts: 78 brendabee User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 6
I use a single directory. Plus, I've had problems with robots.txt files in the past and don't like putting information in a robots.txt file because it gives information to people I don't want to give information to.

Reply With Quote
  #4  
Old May 2nd, 2003, 12:01 AM
yintercept yintercept is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Location: Salt Lake City
Posts: 132 yintercept User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 6
Yeah, I worry about that too. robots.txt screams: "This is where the good stuff is." I only use it for pages I want the search engines to ignore. I simply hope that no one ever puts a link to the admin screens for the site.

As for NO FOLLOWS, I suspect that just tells the robots not to put the links from the page in their list of pages to follows. I suspect that they would end up reading the links if there are other pages with links to your inner pages.

So if you had a format: index -> inner pages.

Putting NO FOLLOW on the index page will not prevent the inner pages from being indexed if there are other pages on the web pointed to your innies.

Reply With Quote
  #5  
Old May 2nd, 2003, 12:04 AM
brendabee's Avatar
brendabee brendabee is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Posts: 78 brendabee User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 6
So the NO FOLLOW applies to the links on the page and NOT the page itself?

Reply With Quote
  #6  
Old May 2nd, 2003, 12:19 AM
fathom's Avatar
fathom fathom is offline
CnR 866-977-8675
Click here for more information.
 
Join Date: Mar 2003
Location: Saskatoon, Saskatchewan, Canada
Posts: 8,897 fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 3 Months 1 Week 2 Days 20 h 53 m 30 sec
Reputation Power: 73
Send a message via ICQ to fathom Send a message via AIM to fathom Send a message via MSN to fathom Send a message via Yahoo to fathom Send a message via Google Talk to fathom Send a message via Skype to fathom Send a message via XFire to fathom
Quote:
Originally posted by "brendabee"

I use a single directory. Plus, I've had problems with robots.txt files in the past and don't like putting information in a robots.txt file because it gives information to people I don't want to give information to.


I really don't see how

User-agent: *
Disallow: /

is giving any information away.

1. no one actually gets your robots.txt unless they type in the exact location e.g. www.domain.com/robots.txt

2. even so -- what security, privacy, copyright, and/or intellectual property issue are of concern with:

User-agent: *
Disallow: /

Am I missing something?
__________________
FREE LINKS for LINKBAIT Catch 'n Re-Lease Me! - We are what we repeatedly do… excellence, then, is not an act, but a habit. — Aristotle

Reply With Quote
  #7  
Old May 2nd, 2003, 12:21 AM
fathom's Avatar
fathom fathom is offline
CnR 866-977-8675
Click here for more information.
 
Join Date: Mar 2003
Location: Saskatoon, Saskatchewan, Canada
Posts: 8,897 fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 3 Months 1 Week 2 Days 20 h 53 m 30 sec
Reputation Power: 73
Send a message via ICQ to fathom Send a message via AIM to fathom Send a message via MSN to fathom Send a message via Yahoo to fathom Send a message via Google Talk to fathom Send a message via Skype to fathom Send a message via XFire to fathom
Also .htaccess is another alternative

Reply With Quote
  #8  
Old May 2nd, 2003, 02:03 AM
simple-biz's Avatar
simple-biz simple-biz is offline
simple-biz.com
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Feb 2003
Location: Thailand, Phuket
Posts: 223 simple-biz User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 h 46 m 37 sec
Reputation Power: 6
Send a message via Yahoo to simple-biz Send a message via Skype to simple-biz
I beleive the best way is to create a new folder and drop all those files in there and use a .htacces or a login module so that only the intended audience can access the files there.

Hope this helps ;)

Reply With Quote
  #9  
Old May 3rd, 2003, 10:37 PM
brendabee's Avatar
brendabee brendabee is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Posts: 78 brendabee User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 6
fathom, thanks for the second post. I don't want to block robots from every page, just the duplicates.

simple-biz, thanks. I think that may be the solution. I was hoping for something simpler (i.e. something that wouldn't require changing code on many of my pages).

Reply With Quote
  #10  
Old May 3rd, 2003, 10:55 PM
theBear's Avatar
theBear theBear is offline
Contributing User
SEO Chat Novice (500 - 999 posts)
 
Join Date: Mar 2003
Location: Maine USA
Posts: 524 theBear User rank is Private First Class (20 - 50 Reputation Level)theBear User rank is Private First Class (20 - 50 Reputation Level) 
Time spent in forums: 25 m 38 sec
Reputation Power: 6
Send a message via AIM to theBear
Re: NO FOLLOW

Quote:
Originally posted by "brendabee"

I am trying to keep SE bots out of some of my pages. What's the best way to do this? I have tried using
<META NAME="robots" content="noindex,nofollow">

Freshbot came through one of my sites and completely ignored it.

I've read that Googlebot ignores meta tags. Is there another way? I'd like to avoid putting it in the robots.txt file.


That noindex tells the bot to not index that page. That means it will remove that page from the index. Don't think that is what you really want.

Cheers,
__________________
theBear

Reply With Quote
  #11  
Old May 4th, 2003, 12:02 AM
brendabee's Avatar
brendabee brendabee is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Posts: 78 brendabee User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 6
Yes. I have several sites that have identical pages due to overlap. I want the correct pages to be indexed on the right sites.

So if Goolgebot visits a page with the metatag on it, it won't index that page? And this won't affect any of the pages linked to from that page if they are linked to from another page such as the home page?

Reply With Quote
Reply

Viewing: SEO Chat ForumsSearch Engine StrategiesSearch Engine Optimization > NO FOLLOW


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 2 hosted by Hostway
Stay green...Green IT