STUPID Altavista

Yep, they can't even parse out all the links to spider or are you talking about Scooter it has the same problem. Wait till you see grub.

Cheers,

What does this mean? (and what means "grub"?)

I wonder, if it also could apply to a problem I have.

I have my site bloggitt.de also submitted through the Express Inclusion, and since two weeks (so from the beginning) they give me error 700. They also say, the problem is on my side - but I cannot figure out what could be wrong.

What especially wonders me, is the "normal" crawling from other spiders...

I'll answer the grub question first---- grub is yet another web crawling robot (243 clients running - crawling 18,958,378 URLs in the last 24 hours). Grub has joined forces with LookSmart to take community web crawling to the next level. Our mission to eventually crawl and assemble the latest state information for every document on the Internet remains unchanged.

There are still issues to be worked out. Folks this puppy should prove interesting.

http://www.grub.org

Now to help with those other questions could you show us the log entries for the crawler that is having problems with your site? A number of crawlers aren't too smart, in fact some are down right stupid.

Cheers,

First thanks for the info about grub. Thats interesting!

On my log entries I find following a lot the last couple of days:
216.39.48.91 bloggitt.de - [02/Apr/2003:05:28:12 +0200] "GET /robots.txt HTTP/1.1" 200 37 "-" "Scooter/3.2"
216.39.48.91 bloggitt.de - [02/Apr/2003:05:28:17 +0200] "GET /gebloggitt70.html HTTP/1.1" 200 7366 "-" "Scooter/3.2"

This must be the regular Scooter, as (to my info) the IP for the Express Submission is 216.39.50.xxx

Nothing to be found on the error log.

From the Express Submission I get each morning the message, the site couldn't be spidered because of Error 700. Once they meant, it's because of the ISO-Code I defined on the top. So I deleted it to test - but no result. The server is not blocking their IP.
I really can't figure out why - especially as the regular Scooter seems to have no problems! (no results to be found on AV, but I guess that still takes some time)


That's what I am beginning to believe reg. the AV EI!

Reg,
Birthe

Scooter has a few problems it doesn't properly parse out the links for retrieval. It results in 404 errors because the page it tries to get isn't a real page Scooter is droping the last part of the urls in the link.

That is why this happens ...

Tue Apr 1 01:10:48 2003] [error] [client 216.39.48.61] File does not exist: /home/dnware/public_html/news/archive/articles/htt

and

[Tue Apr 1 03:18:35 2003] [error] [client 216.39.48.102] File does not exist: /home/virtual/site8/fst/var/www/html/Cata

The last one should read:

/home/virtual/site8/fst/var/www/html/Catalog/

What is an error 700 ???? I haven't run into that one.

Cheers,

Error 700 ist driving me crazy

Their first explanation:

Do you have a log entry of the bot hiting your server?

I had no trouble hiting your site.

Cheers,

Nope.
Apart from the ones I copied above.

does anyone even care about AV anymore? It seems to me like 1/2 their results are just garbage anyway

I don't know. I quite like their combined suggestions - and I expected that they would become more important after their relaunch.

BUT: my last 4 weeks with AV really annoyed me, and although Scooter visits me often - I'm still not listed. And reg. my paid listing: they only got me once, now I receive 1201-Errors: the site is already listed, but couldn't be updated.

If they realy want to make a chance, in my opinion they have to do a lot of work to keep up with their competition.

Birthe,

I do agree, in my book AV is toast. No meat just fluff. Aren't impressed with search engine results either despite recent updates. On every occasion I have paid to submit to Altavista, it hasn't resulted more than a few hits a month. Averaging it out, its about 10.00 a click.
I declined to submit to AV for my education site, which is crawled by a good majority of the major engines out there. Figured it wasn't worth the money.
BUT, I am excited to see the developments of GRUB. I have read that they plan to implement a toolbar so that its a combined community shared crawling. Might give some engines a run for there money at the combined power of the crawling. But I already think of somethings to prevert the use of this toolbar. It comes down to time and money.

Good luck with Av crawling your site.
Haven't even heard of 700 errors. Sorry I couldn't be more assistance.

Ben

Grub is going to face some major problems.

Scope of problem type problems just like every other SE.

Cheers,

keyword: a LOT...AV is not even a 2nd tier engine IMHO anymore. Who uses them really other than maybe for the Babblefish? Actually this brings up a question: what is the breakdown of internet users' use of search engines? meaning google holds what market share, vs the others?

I don't waste my time with AltaVista.

Most of their SERP are irrelevant and who uses them anyways?

With any luck Overture will whip them back into shape but I would not count on this anytime soon.