|
|||||||||
|
|||||||||
|
|||||||||
 |
|||||||||
| |
||
|
|
|
|
|||||||||
|
|||||||||
|
|||||||||
|
|||||||||
| |
||
|
|
|
| |||||||||
 |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
SEO Chat Forums Sponsor:
|
|
|
|
#1
May 13th, 2004, 03:37 PM
|
||||
|
||||
use robot.txt even if site is all .htm?
if you want the engines to search your entire site (all in static .htm) then is it still beneficial to make use of a robot.txt? Thanks...
|
|
#2
May 13th, 2004, 04:01 PM
|
||||
|
||||
|
No. You only need a robots.txt file to block access, not to allow access.
__________________
Military Singles Dating
|
|
#3
May 13th, 2004, 04:29 PM
|
||||
|
||||
|
Actually some spiders do have a fit if a robots file is not present even if you allow full access. They are small and you will never get any major traffic from them but they do exist.
Old buggers.
|
|
#4
June 15th, 2004, 09:20 AM
|
||||
|
||||
robots.txt
Well, this is just to make some spiders happy (no matter big or small SE).
My point is...if this small work (i.e. putting up a robots.txt) makes some spiders happy, what the heck? Let us make 'em feel happy.  You never know...a small traffic from those small SE, might eventually give you some business. Makes sense? Regards WC
__________________
www.zinfi.com www.zinfitech.com www.zinfiprojects.com
|
|
| Viewing: SEO Chat Forums > Search Engine Strategies > Search Engine Optimization > use robot.txt even if site is all .htm? |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
 |
|
|
|
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
