|
|
|||||||||
|
|||||||||
|
|||||||||
| |
||
| |||||||||
![]() |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
#1
|
|||
|
|||
|
A quick question on robots.txt
Until now I have never had a robots.txt file for any of my websites. With the new Yahoo spider I have decided to add a robots.txt file. I read many websites about how to do this, but am a little confused. Basically I want ALL spiders to index ALL pages, is this correct:
User-agent: * Disallow: For anyone that's intertested I found this on Yahoo http://help.yahoo.com/help/us/ysearch/index.html |
|
#2
|
||||
|
||||
|
The other thing you can do of course is to simply not have a robots.txt file...
- Shawn
__________________
Free Keyword Ranking & Keyword Suggestion Tool Search Engine Forums where you are paid to be a user Western Union | eBay |
![]() |
| Viewing: SEO Chat Forums > Search Engines > Yahoo Search Optimization > A quick question on robots.txt |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
|