- Total Members: 263,787
- Threads: 454,027
- Posts: 1,062,421
Great community. Great ideas.
Welcome to SEOChat, a community dedicated to helping beginners and professionals alike in improving their Search Engine Optimization knowledge. Sign up today to gain access to the combined insight of tens of thousands of members.
-
Jan 5th, 2013, 04:13 AM
#1
BSpider refering to IBM IP address.
BSpider is visting with ip address 129.33.201.12. When i checked at 129.33.201.12 IP Address WHOIS | DomainTools.com , It refers to ibm server, i did a traceroute and it led to AT & T.
whose bot is this. It is crawling everything webpage, css, image files many no of times within 1 minute.
-
Jan 5th, 2013, 09:46 AM
#2
You can use robots.txt and .htaccess to prevent it from crawling things you don't want. There are some bandwidth hoggin bots out there that need to be tamed sometimes.
Similar Threads
-
By jawad in forum Analytics Analysis
Replies: 2
Last Post: Oct 1st, 2011, 01:57 AM
-
By Rockem Sockem in forum Analytics Analysis
Replies: 7
Last Post: Jun 22nd, 2009, 08:46 AM
-
By Funky in forum Google Optimization
Replies: 1
Last Post: Jan 15th, 2009, 10:15 AM
-
By ChrisHill in forum Analytics Analysis
Replies: 0
Last Post: Oct 18th, 2007, 10:43 AM
-
By zanet in forum Google Optimization
Replies: 0
Last Post: Feb 7th, 2005, 05:03 PM