Deep Crawl / Freshbot Activity

From what I can see the Deep Crawl pretty much finished today at about 1am US Central Time. I'm seeing the freshbot being EXTREMELY active. This forum has received hundreds of hits (about 500) in the last 24 hours. I'm not complaining

Anyone seeig the same. Does anyone have hits from the Deep Crawl Bot (IP Range 216.xxx) after 1am Central time (GMT -6:00)??

Anyone else seeing a VERY active freshbot (IP Range 64.68.8xx.xxx)??

yes!

after you mentioned that there are two bots, i looked at the IPs.

I used to have the deep crawl bot visiting my site last week.

Yesterday, I was wondering why google is coming all over again. Now I see that it's the freshbot.

Darrin, what exactly is the difference? What can we infer from this?

I would appriciate you speak your expertise

Thank you!!

The "Freshbot" is the spider(s) that continuosly scan the internet for new documents and updated documents. The "Freshbot" will then usually make changes to the Google index based on what new information is finds. This allows for a slightly fresher index.

The "Deepbot" does it's rounds about once per month, and any information found by the "Deepbot" will be permanently inserted into the google index in the following update.

If you get hit by the Freshbot then there is no guarantee that you will allocated a PR .. If you get hit by the "Deepbot", then the chances are that the next update will bring newly updated PR for you (although note that the PR may very well be the same as that in the last update).

Deepbot = Permanent
Freshbot = Temporary

I got 75 hits from the freshbot in the last 3 hours, so it is definetly going crazy. It ususally doesnt hit that rapidly. I would usually get 100 hits in a 24 hour time period. But i did put a bunch of new content up on the site, so im definetly not complaining

-Josh

This forum has got about 300 hits in the last 3 hours .. weird .. They must be really at max capacity in terms of bandwidth is everyone is getting hit this hard.

Google is using a different IP nearly every time it hits the site -- so it's giving me the impression that I have about twice the traffic that I really do

By the way : Don't get me wrong, I love the extra hits from the Googlebot .. All other spiders are welcome as well..

PS - If you have your site listed in your profiles then you might be getting hit b ythe googlebot hardish as well I know they are hitting the profiles!

Just wanted to say I noticed the same for the deep-crawling ending, and freshbot going insane. Quick question though, how long after the freshbot visits should we see updates on Google? Thanks.

The stuff indexed by the freshbots usually show up 3-5 days after indexed. Atleast thats what it seems like. Ahh now i have to wait for my stuff to get in the index. Yay.... Well not really.

-Josh

Could it be that if I miss the deep crawl then my site receive less visits from Freshbot??..before the last dance my site was hit by freshbot every 3-4 days and it's not the same now..