Search Engine Spiders
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
Go Back   SEO Chat Forums > Search Engine Strategies > Search Engine Spiders
Reload this Page

.htaccess code to block all the unwanted rebots?

Discuss .htaccess code to block all the unwanted rebots? in the Search Engine Spiders forum on SEO Chat.
.htaccess code to block all the unwanted rebots? Search Engine Spiders forum discussing how to know which search engines are visiting your site by identifying their spiders. Share their IPs and spidering habits, and plan for them to get better SEO results.

Record onscreen activity, your voice & webcam video. Turn everyday presentations into HD-quality videos.

For perfectly clear screencasts every time.

We cover the world of search engine optimization like no one else, constantly updating you with the best information and news from around the world.

Take your business to the next level, with "The Edge of Success: 9 Building Blocks to Double Your Sales." Inside, you will discover how to easily capture more leads, get repeat sales, and significantly grow your business.


Try Camtasia Studio today!
Sign Up today!
Download this FREE ebook now!

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old June 27th, 2003, 07:58 PM
Nintendo's Avatar
Nintendo Nintendo is offline
King of da Wackos
SEO Chat Novice (500 - 999 posts)
  
Join Date: Jan 2003
Location: Planet Zeekois
Posts: 680 Nintendo User rank is Private First Class (20 - 50 Reputation Level)Nintendo User rank is Private First Class (20 - 50 Reputation Level) 
Time spent in forums: 1 Day 2 h 41 m 30 sec
Reputation Power: 8
.htaccess code to block all the unwanted rebots?
Does any one have a .htaccess code list to block all the unwanted rebots, like ones that collect E-mail address???
Reply With Quote
  #2  
Old June 28th, 2003, 12:45 AM
2K's Avatar
2K 2K is offline
Professional SEO
SEO Chat Novice (500 - 999 posts)
  
Join Date: May 2003
Location: Finland
Posts: 711 2K User rank is Lance Corporal (50 - 100 Reputation Level)2K User rank is Lance Corporal (50 - 100 Reputation Level)2K User rank is Lance Corporal (50 - 100 Reputation Level) 
Time spent in forums: 3 Days 1 h 58 m 54 sec
Reputation Power: 7
Here's some code I use. It requires mod_rewrite:

RewriteEngine On
Options +FollowSymLinks

RewriteCond %{HTTP_USER_AGENT} Wget [OR]
RewriteCond %{HTTP_USER_AGENT} teleport [OR]
RewriteCond %{HTTP_USER_AGENT} go-ahead-got-it [OR]
RewriteCond %{HTTP_USER_AGENT} EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} EmailWolf
RewriteRule ^.*$ /errors/badboy.html [L]
Reply With Quote
Reply

Viewing: SEO Chat Forums > Search Engine Strategies > Search Engine Spiders > .htaccess code to block all the unwanted rebots?

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump




 Free IT White Papers!
 
Create the Optimal Architecture for your Critical Applications
Warburton's the largest independently owned bakery in the UK faced a number of difficult challenges in providing the most robust yet efficient IT infrastructure for their organization's success. IBM's services combined with their xSeries servers created the perfect platform for their SAP environment with sufficient flexibility, and did so in very time effective fashion.
Request Your Free Technology Downloads!
 
Five Best Practices for Deploying a Successful Service-Oriented Architecture
This white paper describes the benefits you can expect with SOA, and how IBM can help take your business there.
Request Your Free Technology Downloads!
 
Gartner Magic Quadrant for Application Delivery Controllers
Gartner summarizes its view on Application Delivery Controllers, evaluates strengths and weaknesses of solutions, and provides Magic Quadrant reporting for a quick comparison across all vendors. Learn from Gartner how you can benefit from an all-in-one device like Citrix NetScaler that delivers the highest levels of availability, performance and security.
Request Your Free Technology Downloads!
 
Knowledge is Power
What you don't know can hurt you, and is likely costing you money and increasing your security risks during an era of scarce resources. This white paper proposes six key strategies that enterprise security managers can use to improve their network defense posture.
Request Your Free Technology Downloads!
 
Rationalizing the Multi-Tool Environment
The rationalized multi-tool approach is flexible, scalable and cost effective. It provides the necessary input to the IT service management business processes. It preserves prior investments in monitoring tools, empowers technologists to select the best tools with which to do their jobs, and enhances effective response to incidents.
Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 




© 2003-2010 by Developer Shed. All rights reserved. DS Cluster 9 Hosted by Hostway. 
For more Enterprise Application Development news, visit eWeek