|
|||||||||
|
|||||||||
|
|||||||||
 |
|||||||||
| |
||
|
|
|
|
|||||||||
|
|||||||||
|
|||||||||
|
|||||||||
| |
||
|
|
|
| |||||||||
 |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
SEO Chat Forums Sponsor:
|
|
|
|
#1
February 19th, 2004, 06:07 AM
|
|||
|
|||
|
A quick question on robots.txt
Until now I have never had a robots.txt file for any of my websites. With the new Yahoo spider I have decided to add a robots.txt file. I read many websites about how to do this, but am a little confused. Basically I want ALL spiders to index ALL pages, is this correct:
User-agent: * Disallow: For anyone that's intertested I found this on Yahoo http://help.yahoo.com/help/us/ysearch/index.html
|
|
#2
February 19th, 2004, 01:29 PM
|
||||
|
||||
|
The other thing you can do of course is to simply not have a robots.txt file...
- Shawn
__________________
Free Keyword Ranking & Keyword Suggestion Tool Search Engine Forums where you are paid to be a user Western Union | eBay
|
|
| Viewing: SEO Chat Forums > Search Engines > Yahoo Search Optimization > A quick question on robots.txt |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
 |
|
|
|
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
