SEO Scripts
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
 
User Name:
Password:
Remember me
Go Back   SEO Chat Forums > Other > SEO Scripts
Reload this Page

Robots question

Discuss Robots question in the SEO Scripts forum on SEO Chat.
Robots question SEO Scripts forum discussing PHP, Perl, CGI, ASP, JavaScript, redirects, robots.txt and other code. Post questions about optimizing different kinds of code, and somebody might be able to help.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old December 27th, 2007, 04:10 AM
Glenn Kilpatric Glenn Kilpatric is offline
Contributing User
SEO Chat Novice (500 - 999 posts)
  
Join Date: Jun 2006
Location: England
Posts: 583 Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level)Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level)Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level)Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 4 Days 12 h 35 m 46 sec
Reputation Power: 4
Robots question
When dealing with php actions in a forum is the code at the bottom of the page enough to deal with google listings like

www.whitbyseaanglers.co.uk/.../saltburn-christmas-open/0/?action=ignore;u=156;msg=22625

Code:
User-agent: *
Disallow: /*?action=ignore
Reply With Quote
  #2  
Old December 27th, 2007, 04:17 AM
fathom's Avatar
fathom fathom is offline
CnR 866-977-8675
Click here for more information.
  
Join Date: Mar 2003
Location: Saskatoon, Saskatchewan, Canada
Posts: 8,897 fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 3 Months 1 Week 2 Days 20 h 53 m 30 sec
Reputation Power: 73
Send a message via ICQ to fathom Send a message via AIM to fathom Send a message via MSN to fathom Send a message via Yahoo to fathom Send a message via Google Talk to fathom Send a message via Skype to fathom Send a message via XFire to fathom
Correct!
__________________
FREE LINKS for LINKBAIT Catch 'n Re-Lease Me! - We are what we repeatedly do… excellence, then, is not an act, but a habit. — Aristotle
Reply With Quote
  #3  
Old December 27th, 2007, 07:41 AM
Jean-Luc Jean-Luc is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Dec 2004
Location: Brussels, Belgium
Posts: 394 Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 51 m 57 sec
Reputation Power: 7
The asterisk * is not a special character according to the robots.txt standard. It is understood by Googlebot though.

Jean-Luc
__________________
AWStats Support : add-on's, extra sections, forum, installation assistance
AWStats remote service for less than $2 a month
Checking redirects is now as easy as 1 2 3, even if you are not a HTTP-header guru !
Reply With Quote
  #4  
Old December 27th, 2007, 10:29 AM
fathom's Avatar
fathom fathom is offline
CnR 866-977-8675
Click here for more information.
  
Join Date: Mar 2003
Location: Saskatoon, Saskatchewan, Canada
Posts: 8,897 fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 3 Months 1 Week 2 Days 20 h 53 m 30 sec
Reputation Power: 73
Send a message via ICQ to fathom Send a message via AIM to fathom Send a message via MSN to fathom Send a message via Yahoo to fathom Send a message via Google Talk to fathom Send a message via Skype to fathom Send a message via XFire to fathom
Quote:
Originally Posted by Jean-Luc
The asterisk * is not a special character according to the robots.txt standard. It is understood by Googlebot though.

Jean-Luc


To be more accurate - when they agree to support sitemaps they agreed to the same standard for robots.txt...

Therefore Google, Yahoo and MSN all support (*) wildcards like *.gif
Reply With Quote
  #5  
Old December 27th, 2007, 11:30 AM
Jean-Luc Jean-Luc is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Dec 2004
Location: Brussels, Belgium
Posts: 394 Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level)Jean-Luc User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 51 m 57 sec
Reputation Power: 7
You are right that Google, Yahoo and MSN all implemented a proprietary extension to support wildcards, but even so, this doesn't change the standard. People should be aware that many robots and search engines respecting robots.txt don't support wildcards.

They follow the original independent specification for robots.txt, available at www.robotstxt.org.

Jean-Luc
Reply With Quote
  #6  
Old December 27th, 2007, 12:30 PM
fathom's Avatar
fathom fathom is offline
CnR 866-977-8675
Click here for more information.
  
Join Date: Mar 2003
Location: Saskatoon, Saskatchewan, Canada
Posts: 8,897 fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level)fathom User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 3 Months 1 Week 2 Days 20 h 53 m 30 sec
Reputation Power: 73
Send a message via ICQ to fathom Send a message via AIM to fathom Send a message via MSN to fathom Send a message via Yahoo to fathom Send a message via Google Talk to fathom Send a message via Skype to fathom Send a message via XFire to fathom
Quote:
Originally Posted by Jean-Luc
You are right that Google, Yahoo and MSN all implemented a proprietary extension to support wildcards, but even so, this doesn't change the standard. People should be aware that many robots and search engines respecting robots.txt don't support wildcards.

They follow the original independent specification for robots.txt, available at www.robotstxt.org.

Jean-Luc


When 92% of the searching market use Google, Yahoo, Msn (which include AOL, Netscape, Alta Vista, Lycos, Hotbot, and Go)... the standard is absolute wrong... and must therefore be ignored until they get off their asses and actually do what they are suppose to do... update the standard for the 21st centry.

Let's look at this in context... the robots.txt is for exclusion - right?

User-agent: (each not excepting wildcards)
disallow: /

Problem solved!
Comments on this post
dzine agrees: LOL Good riddance
Reply With Quote
  #7  
Old December 27th, 2007, 04:55 PM
Glenn Kilpatric Glenn Kilpatric is offline
Contributing User
SEO Chat Novice (500 - 999 posts)
  
Join Date: Jun 2006
Location: England
Posts: 583 Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level)Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level)Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level)Glenn Kilpatric User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 4 Days 12 h 35 m 46 sec
Reputation Power: 4
Cheers Guys. Glad thats sorted. I added an ignore option to my forum because some of the members are behaving like kids

I got up today and I had over 100 listings for ignore messages.

Google only gives me about 120 listings in the non supplememntal index for the forum so I can afford to loose most of them to rubbish that wont bring any traffic in.
Reply With Quote
Reply

Viewing: SEO Chat Forums > Other > SEO Scripts > Robots question

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 4 hosted by Hostway. 
Stay green...Green IT