How to track Google Crawling?

Read somewhere," yesterday, google crawled deep into my website...". How can google crawling into a website be tracked? Any software or tool needed? Help please...

Darren develop this to email when Googlebot crawls - SpyderTrax - NEW Robot Tracking Script

<edit>Threads merged</edit>

So spider trax can track a spider crawling. But how can I manipulate this if at all I can track? And by the way, where can I get the Spidertrax from?Help please....:-?

Spider Trax

Not sure what else you are asking

What I mean is if I can track google crawling, can I use it in some way to improve my site's ranking? After all, what's the fun of knowing this. May be the question sounds funny, but it's vexatious to me. My webmaster wants to know why should we use the program. So do I. "Enlighten please!"

No that's it - it just notifies you that googlebot is crawling your site.

Maneesh - please post common topics in the same thread rather than starting a new thread each time you post.

Thanks

Knowing how often Googlebot visits and which page it visits teaches you a little about robot behaviour. e.g. which pages does it visit more often and can you change that behaviour, say, by updating more often?

It isn't "fun" but, to the contrary, serious "webmasters" feel miserable when the robot has not visited in a while. If you like fun then don't worry about robots and fly by the seat of your pants.

I am not convinced that Spyder Trax accurately picks up all the robots in its hard coded list but it works, and when you run it on a couple of sites you can compare robot visits and see which URLs need to be submitted.

Thanks Ash! (If that's your name). This question troubled me for a long time!:smile:

And sorry fathom. In fact I was under the impression that new threads are given priority. :-?

Thanks to everyone!:smile:
-Maneesh.

And new posts bump threads to the top -- thus the same thing with the exception that if someone else ever had all the same questions they would only need to open 1 thread insteand of 10 to find all the answers. ;)