|
|
|||||||||
|
|||||||||
|
|||||||||
| |
||
| ||||||||||||||||||||||||||
![]() |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
#1
|
|||
|
|||
|
Checking for robots?
How often do you people look at your logs for robost in general and googlebot in particular? Is there any idea to check them logs very often?
Also, can anyone suggest any nice proggie for looking through and sorting info from the raw server logs? I'd appreciate it if you could point me to some freeware proggie or something. |
|
#2
|
||||
|
||||
|
There is something effective that I have looked at recently. On one of my sites, I do not have a robots.txt file. In order to see the robots that crawled the site that day I look at the error log. Requests for 'robots.txt' all in one page, and include an IP. Yet, its hard to determine what bot it is by looking at the IP. I have a list of googlebot IP's but others not.
Might be helpful. Darrin has a great program for tracking Googlebot hits on a his site called: GoogleTrax. You might check into that.
__________________
Rank Smart Search Marketing - Search Engine Roundtable - Search Marketing Association - North America |
|
#3
|
|||
|
|||
|
If the server-ware you are using is Apache then "report magic 2.01" is a great bit of software, it'll let you see what search engine keywords your getting hits for as well! I think its free
__________________
|
![]() |
| Viewing: SEO Chat Forums > Other > HTML Coding > Checking for robots? |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
|
|