February 12th, 2003, 11:19 AM
|
 |
Contributing User
|
|
Join Date: Jan 2003
Location: No. Cal.
Posts: 67
Time spent in forums: < 1 sec
Reputation Power: 10
|
|
|
do deepcrawl / freshbot work together?
I'm looking over yesterday's google visits to my site, and I see mostly freshbot visiting the entryway pages to the new sections I had hoped the deepcrawl would visit. do the two work in tandem, with freshbot tipping deepcrawl off to areas to check out on its next visit? There are 1,000s of new pages I am hoping to have deepcrawl visit, but so far, freshbot has only visited them, and it hasn't done a very thorough job of indexing the directories (follows only 1 link to new page, then stops).
I've done away with SIDs, and straightened out the php URLs, so everything should be good to go.
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
