What The Heck?

I just created a site for a client and Im trying to add it to altavista - look at the error message there giving me:

Review your selection before proceeding to billing.

URL http sniped by clasione
Status Rejected : Retrieval of robots.txt leads to a redirect.

--------------------------------------------------------------------------------

I dont have a robots.txt file? what the heck?

Weird!

Rich, my partner, who is smarter than me says that
when you enter:
http://guide2buyingahome.com/robots.txt
you get a 404 -this may be the redirect they refer to. Odd, it looks like they require a robots text file now?

Try putting up a robots.txt file and see what happens, good example
http://www.cnn.com/robots.txt

Just a guess

This is now confermed -

I added a robots.txt and it excepted me....

You now must Have a Robots.txt fill to submit to altavista. This is definetly new starting sometime after February

Well, I guess it makes sense. If you look at the files CNN.com/robots.txt disallows, they don't really have content that is helpful to the collecting of data. This was probably intended to make the robots more efficient and not caught up in odd places like the CSS or external javascript.

I wonder if this is a trend?