Page 1 of 6 123 ... Last
  • Jump to page:
    #1
  1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2002
    Posts
    29
    Rep Power
    0

    Cool script: Emails you when googlebot hits your domain (*)


    Just something quick I wrote that I thought Id share...

    Basically means that you dont have to check your logs to see if google is about... As soon as freshbot or deepbot crawls your site - you get mail

    I wouldnt put this on too many pages or you will get an email every time a page gets hit...

    You will need PHP of course...

    http://www.phphacks.com/googlebot.php

    If you make any enhancements please share em

    At some stage I might add some kind of logging function...
  2. #2
  3. Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2002
    Posts
    77
    Rep Power
    17
    Cool php script - this is what I use in coldfusion (I stick this in the application.cfm)

    <cfif #cgi.HTTP_USER_AGENT# contains "google" or
    #cgi.HTTP_USER_AGENT# contains "URL_Spider_SQL" or
    #cgi.HTTP_USER_AGENT# contains "Firefly" or
    #cgi.HTTP_USER_AGENT# contains "NationalDirectory" or
    #cgi.HTTP_USER_AGENT# contains "Ask Jeeves" or
    #cgi.HTTP_USER_AGENT# contains "TECNOSEEK" or
    #cgi.HTTP_USER_AGENT# contains "InfoSeek" or
    #cgi.HTTP_USER_AGENT# contains "WebFindBot" or
    #cgi.HTTP_USER_AGENT# contains "girafabot" or
    #cgi.HTTP_USER_AGENT# contains "crawler" or
    #cgi.HTTP_USER_AGENT# contains "www.galaxy.com" or
    #cgi.HTTP_USER_AGENT# contains "Googlebot" or
    #cgi.HTTP_USER_AGENT# contains "Scooter" or
    #cgi.HTTP_USER_AGENT# contains "Slurp" or
    #cgi.HTTP_USER_AGENT# contains "appie" or
    #cgi.HTTP_USER_AGENT# contains "FAST" or
    #cgi.HTTP_USER_AGENT# contains "fast" or
    #cgi.HTTP_USER_AGENT# contains "WebBug" or
    #cgi.HTTP_USER_AGENT# contains "Spade" or
    #cgi.HTTP_USER_AGENT# contains "ZyBorg" or
    #cgi.HTTP_USER_AGENT# contains "rabaz"
    >
    <cfmail to="admin@why2kit.com" from="admin@why2kit.com" subject="A Spider Is Crawling" type="html">
    Refer - #cgi.HTTP_REFERER#<br>
    agent- #cgi.HTTP_USER_AGENT#<br>
    </cfmail>

    Hope this is of assistance to someone

    Cheers

    Richard
  4. #3
  5. Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2002
    Posts
    36
    Rep Power
    0
    I hope that I've done that little one right...

    I popped it at the top of a PHP page (so there was already a '< ? PHP and ? > tags (without the spaces)' )

    I can't see it when I look at the page source (so I know I'm doing somthing right) If I've already missed the spider, I'll not know until next month now, I shouldn't have missed the spider already (I hope) it's a new site, so I don't yet have a PR, so I guess I'll not get spidered until late on in the sequence.
  6. #4
  7. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2002
    Posts
    29
    Rep Power
    0
    If you want to test the above script to see if it works, then change the word googlebot to mozilla (will work for 90% of browsers) and then load your page - you should get an email... If you do - it works.. Switch it back to googlebot and your away
  8. #5
  9. Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2002
    Posts
    36
    Rep Power
    0
    Angus you are a *

    I got a mail from Apache@my server

    Cool!
  10. #6
  11. Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Dec 2002
    Posts
    77
    Rep Power
    17
    If you want to emulate an agent viewing a page you can use my spiderview tool at http://www.y2kinternet.com/spiderview.cfm
  12. #7
  13. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2003
    Location
    Somewhere in the EU
    Posts
    140
    Rep Power
    17

    coooooool


    Wow,

    nice script! I use CFML too, but since now I never had an idea like this.

    I think I'll use this lovely script soon.


    I'm looking forward to learn some hot tips in this forum :-)

    Urs,
    Innuendo
  14. #8
  15. Moderator
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jan 2003
    Location
    Madrid, Spain
    Posts
    1,382
    Rep Power
    18
    Angus - great script, but would it be possible to adapt it to differentiate between the Googlebot and the Freshbot ? (I can send you a list of almost all IP's of these bots).

    I'm more interested in knowing when the Googlebot is around and doing its deep crawl for the next update than the Freshbot, which hits my site just about every day.

    Thanks, Gringo
  16. #9
  17. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2003
    Location
    Somewhere in the EU
    Posts
    140
    Rep Power
    17

    sure!


    Sure its possible to make a difference between the freshbot.

    Send me the list of the different IPs and I'll post the new script.

    If its ok for all in here. It's not my personal script.

    sl,
    Innuendo
  18. #10
  19. Moderator
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jan 2003
    Location
    Madrid, Spain
    Posts
    1,382
    Rep Power
    18
    Freshbot's IP starts with 64 and Googlebot's with 216 ...

    216.239.33.5 ... proxy.google.com
    216.239.35.4 ... 216.239.35.4
    216.239.35.5 ... proxy.google.com
    216.239.39.5 ... 216.239.39.5
    216.239.45.4 ... 216-239-45-4.google.com
    216.239.46.100 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.101 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.102 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.103 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.104 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.105 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.106 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.113 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.116 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.118 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.12 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.121 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.124 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.128 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.13 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.133 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.134 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.140 ... crawl5.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.141 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.147 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.148 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.149 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.151 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.152 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.153 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.154 ... crawl6.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.163 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.164 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.165 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.170 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.171 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.172 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.173 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.176 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.180 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.182 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.184 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.186 ... crawl7.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.19 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.191 ... crawl8.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.193 ... crawl8.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.20 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.200 ... crawl8.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.202 ... crawl8.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.204 ... crawl8.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.210 ... crawl8.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.22 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.220 ... crawl9.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.222 ... crawl9.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.223 ... crawl9.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.226 ... crawl9.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.23 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.235 ... crawl9.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.236 ... crawl9.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.239 ... crawl9.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.26 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.27 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.3 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.30 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.36 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.39 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.41 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.42 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.47 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.48 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.55 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.58 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.60 ... crawl2.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.63 ... crawl3.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.66 ... crawl3.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.7 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.74 ... crawl3.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.76 ... crawl3.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.77 ... crawl3.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.79 ... crawl3.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.8 ... crawl1.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.80 ... crawl3.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.82 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.85 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.87 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.88 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.89 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.90 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.92 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.96 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.98 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    216.239.46.99 ... crawl4.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.10 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.12 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.13 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.14 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.15 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.16 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.17 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.18 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.19 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.25 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.26 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.27 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.28 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.30 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.31 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.32 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.33 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.34 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.35 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.36 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.37 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.38 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.39 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.4 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.41 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.43 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.44 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.45 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.46 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.47 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.48 ... 64.68.82.48 ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.48 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.49 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.5 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.50 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.51 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.52 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.53 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.54 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.55 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.56 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.57 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.58 ... crawler11.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.6 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.63 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.65 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.66 ... 64.68.82.66 ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.66 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.67 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.68 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.69 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.7 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.70 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.71 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.72 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.73 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.74 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.75 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.76 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.77 ... 64.68.82.77 ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.77 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.78 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.79 ... crawler12.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.82.8 ... crawler10.googlebot.com ... Googlebot/2.1 (+http://www.googlebot.com/bot.html)
    64.68.86.10 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.14 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.165 ... crawler7.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.17 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.170 ... crawler7.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.18 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.2 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.24 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.27 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.29 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.31 ... crawler2.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.33 ... crawler2.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.42 ... crawler2.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.46 ... crawler2.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.74 ... crawler3.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.80 ... crawler3.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
    64.68.86.9 ... crawler1.googlebot.com ... Googlebot-Image/1.0 (+http://www.googlebot.com/bot.html)
  20. #11
  21. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2003
    Location
    Somewhere in the EU
    Posts
    140
    Rep Power
    17

    so: here it is!


    Here is the new CFML-Script:

    <cfif #cgi.HTTP_USER_AGENT# contains "google" or
    #cgi.HTTP_USER_AGENT# contains "URL_Spider_SQL" or
    #cgi.HTTP_USER_AGENT# contains "Firefly" or
    #cgi.HTTP_USER_AGENT# contains "NationalDirectory" or
    #cgi.HTTP_USER_AGENT# contains "Ask Jeeves" or
    #cgi.HTTP_USER_AGENT# contains "TECNOSEEK" or
    #cgi.HTTP_USER_AGENT# contains "InfoSeek" or
    #cgi.HTTP_USER_AGENT# contains "WebFindBot" or
    #cgi.HTTP_USER_AGENT# contains "girafabot" or
    #cgi.HTTP_USER_AGENT# contains "crawler" or
    #cgi.HTTP_USER_AGENT# contains "www.galaxy.com" or
    #cgi.HTTP_USER_AGENT# contains "Googlebot" or
    #cgi.HTTP_USER_AGENT# contains "Scooter" or
    #cgi.HTTP_USER_AGENT# contains "Slurp" or
    #cgi.HTTP_USER_AGENT# contains "appie" or
    #cgi.HTTP_USER_AGENT# contains "FAST" or
    #cgi.HTTP_USER_AGENT# contains "fast" or
    #cgi.HTTP_USER_AGENT# contains "WebBug" or
    #cgi.HTTP_USER_AGENT# contains "Spade" or
    #cgi.HTTP_USER_AGENT# contains "ZyBorg" or
    #cgi.HTTP_USER_AGENT# contains "rabaz"
    >

    <cfif left(CGI.Remote_ADDR, 2) eq '21' and cgi.HTTP_USER_AGENT contains 'google'>
    <cfset i_am = 'I am: Googlebot'>
    <cfelseif left(CGI.Remote_ADDR, 2) eq '64' and cgi.HTTP_USER_AGENT contains 'google'>
    <cfset i_am = 'I am: Freshbot'>
    <cfelse>
    <cfset i_am = ''>
    </cfif>

    <cfmail to="admin@why2kit.com" from="admin@why2kit.com" subject="A Spider Is Crawling" type="html">
    Refer - #cgi.HTTP_REFERER#<br>
    agent- #cgi.HTTP_USER_AGENT#<br>
    #i_am#
    </cfmail>





    Try this - i think it works :-)

    Urs, Innuendo
  22. #12
  23. Moderator
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jan 2003
    Location
    Madrid, Spain
    Posts
    1,382
    Rep Power
    18
    Thanks, but ... er ... any chance of having it in PHP?
  24. #13
  25. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2003
    Location
    Somewhere in the EU
    Posts
    140
    Rep Power
    17

    CFML rulez the world


    Hm, actually not - maybe i'll make this in php - but I need some time for it.

    You don't have a high-quality server, which supports CFML?

    So, if I have some time, i'll test this with php.

    Urs,
    Innuendo
  26. #14
  27. Moderator
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jan 2003
    Location
    Madrid, Spain
    Posts
    1,382
    Rep Power
    18

    Re: CFML rulez the world


    Originally posted by "Innuendo"

    You don't have a high-quality server, which supports CFML?
    Never tried it. My server supports PHP and mySQL, so it may also support CFML - I'll try and see. Actually I wanted to include the PHP script in a normal page using SSI - I suppose this would also work for CFML ?
  28. #15
  29. Contributing User
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Jan 2003
    Location
    Somewhere in the EU
    Posts
    140
    Rep Power
    17

    cfml...


    Re-hi,

    no, i don't think that your server supports cfml. Coldfusion is not free, so only few ISP have a Coldfusion Server installed.

    You can also include .CFM pages per SSI - but only, if your server supports cfml.

    Innuendo
Page 1 of 6 123 ... Last
  • Jump to page:

Similar Threads

  1. expired domain script
    By clasione in forum Web Design, Coding and Programming
    Replies: 7
    Last Post: Apr 22nd, 2010, 05:53 AM
  2. Has anyone found a script to track Googlebot through a website?
    By WhiteGyr in forum Google Optimization
    Replies: 19
    Last Post: Jan 23rd, 2005, 08:23 AM
  3. Domain Names and Legal Issues
    By clasione in forum SEO Help (General Chat)
    Replies: 8
    Last Post: Feb 20th, 2004, 01:23 AM
  4. How many hits from Googlebot do you normally get during
    By iamcanadian in forum Google Optimization
    Replies: 10
    Last Post: Apr 23rd, 2003, 09:08 AM
  5. Googlebot php script = .txt log?
    By Nintendo in forum Google Optimization
    Replies: 9
    Last Post: Feb 23rd, 2003, 11:56 AM

IMN logo majestic logo threadwatch logo seochat tools logo