Please HELP! - Log Files!

Hi Everyone - hope the dance has been a happy one for you!
Just a quick request for some information. I am very new to all of this and would really appreciate people's advice as to the best tools for reading web site logs. There is soooooooo much advertising out there as to which is the best, which is not etc.. I thought that the very best people to ask this question of is you guys!
So, please, let me know what software, tools etc you think are the best for reading and analysing your logs. One of my main interest is of course, extracting info of Google's activities on my sites.
Thanks everyone and have a good day.
Scott.

There is no substitute for manual revision of your RAW log files.

I have seen a few low file analyzers but none of them have given me the wealth of information that I get from looking through my logs manually.

Normally when I want to look at Googlebots activity for my site I do the following:

1. Log into my server via SSH (or TelNet).
2. Find my raw access log, which in my case is called "access_log"
3. While in the same directory as the log file I perform this shell command

This will create a file on your server called 'google.txt' and in that file there will be every hit from the Googlebot.

In the code above '.googlebot.com' is the name of Googles remote host. If your server does not perform a DNS lookup (some don't) then you can change this to 'Googlebot' which will track the Googlebot by its HTTP_REFERER.

While we're on the topic, I was just playing around with WebTrends, since that is what my company had been using. My opinion is that it sucks for doing the kind of analysis I need to for SEO. for example, I have no way of clearly distinguishing hits from freshbot from those of deepbot.

Has anyone else run into this? Does everyone pretty much do what Darrin does?

-Nate

Thanks Darrin,
That has answered my question perfectly. I have been doing the manual thing but was lacking the knowledge of how to extract only the Google activity for example. Your reply has answered that for me.
Keep up the great work and I love this forum!!
Regards,
Scott.