MSN Search Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
Go Back   SEO Chat Forums > Search Engines > MSN Search Optimization
Reload this Page

Robots.txt and msnbot

Discuss Robots.txt and msnbot in the MSN Search Optimization forum on SEO Chat.
Robots.txt and msnbot MSN Search Optimization forum discussing the Microsoft search engine for MSN and Live.com. Find tips to help you get high rankings in Microsoft's search. Discuss what factors are most important to its algorithm.

Record onscreen activity, your voice & webcam video. Turn everyday presentations into HD-quality videos.

For perfectly clear screencasts every time.

We cover the world of search engine optimization like no one else, constantly updating you with the best information and news from around the world.

Take your business to the next level, with "The Edge of Success: 9 Building Blocks to Double Your Sales." Inside, you will discover how to easily capture more leads, get repeat sales, and significantly grow your business.


Try Camtasia Studio today!
Sign Up today!
Download this FREE ebook now!

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old January 13th, 2006, 08:33 AM
Kingpin Kingpin is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Nov 2005
Posts: 58 Kingpin User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 20 h 3 m 48 sec
Reputation Power: 5
Robots.txt and msnbot
Hi
Í noticed lately that MSNBOT was trying to do a GET on my robots.txt file, and since I dont have one, ..404 error..so i decided to put one in since I read that robots.txt is important for msnbot..why I dont know, since all this file does is tell the crawler not to crawl certain faolders...
here is what Jennifer Sullivan Cassidy has to say:

MSN’s search engine robot is called MSNbot. The MSNbot has quite a voracious appetite for spidering websites. Some webmasters love it and try to feed it as much as possible. Other webmasters don't see any reason to use up bandwidth for a search engine that doesn't bring them traffic. Either way, MSNbot will not spider your website unless you have the robots.txt. Once it finds your robots.txt, it will wander the site, almost timidly at first. Then MSNbot builds up courage and indexes files rapidly. So much so, that use of the crawl-delay directive is recommended with this robot.
Reply With Quote
  #2  
Old January 13th, 2006, 12:48 PM
kichus's Avatar
kichus kichus is offline
My SE ran out of Gas...
SEO Chat Beginner (1000 - 1499 posts)
  
Join Date: Sep 2004
Location: Bangalore - India
Posts: 1,073 kichus User rank is Corporal (100 - 500 Reputation Level)kichus User rank is Corporal (100 - 500 Reputation Level)kichus User rank is Corporal (100 - 500 Reputation Level)kichus User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 3 Weeks 16 h 46 m 58 sec
Reputation Power: 9
Send a message via MSN to kichus Send a message via Yahoo to kichus
couldn't resist....
__________________
KICHUS - SEO KiD
"Sooner or later, those who win are those who believe they can."
  • Virtue SEO - "a plan with the virtue of being practical"
SEO Consultant India
Reply With Quote
  #3  
Old January 17th, 2006, 07:01 AM
totalit#051's Avatar
totalit#051 totalit#051 is offline
Contributing User
SEO Chat Novice (500 - 999 posts)
  
Join Date: Jul 2005
Location: Botosani, Romania
Posts: 662 totalit#051 User rank is Corporal (100 - 500 Reputation Level)totalit#051 User rank is Corporal (100 - 500 Reputation Level)totalit#051 User rank is Corporal (100 - 500 Reputation Level)totalit#051 User rank is Corporal (100 - 500 Reputation Level)  Folding Points: 1365 Folding Title: Novice Folder
Time spent in forums: 4 Days 12 h 11 m 19 sec
Reputation Power: 7
Send a message via Yahoo to totalit#051
I have proves of sites that were crawled and indexed without a robots.txt ... so if you have one good, if you don't the nly disadvantage is that you error log will show some 404 errors - that's all.
__________________
Epureanu Bogdan On-line - my personal website
Total IT Professional - my company website
Midnight's angel - a website where i help people
Reply With Quote
  #4  
Old January 21st, 2006, 05:27 AM
dburdon's Avatar
dburdon dburdon is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Oct 2005
Location: Kent, UK.
Posts: 326 dburdon User rank is Corporal (100 - 500 Reputation Level)dburdon User rank is Corporal (100 - 500 Reputation Level)dburdon User rank is Corporal (100 - 500 Reputation Level)dburdon User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 20 h 48 m 3 sec
Reputation Power: 7
Robots.txt
Don't have robots.txt on a number of sites with No.1s on MSN.
Reply With Quote
  #5  
Old January 23rd, 2006, 02:33 PM
Kingpin Kingpin is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Nov 2005
Posts: 58 Kingpin User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 20 h 3 m 48 sec
Reputation Power: 5
cant figure out where she got this quote from...doesnt make sense...
Reply With Quote
  #6  
Old January 23rd, 2006, 04:01 PM
dburdon's Avatar
dburdon dburdon is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Oct 2005
Location: Kent, UK.
Posts: 326 dburdon User rank is Corporal (100 - 500 Reputation Level)dburdon User rank is Corporal (100 - 500 Reputation Level)dburdon User rank is Corporal (100 - 500 Reputation Level)dburdon User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 20 h 48 m 3 sec
Reputation Power: 7
Client
I had a client pick up this story. Wondered how he was doing so well on MSN without Robots.txt.
Reply With Quote
Reply

Viewing: SEO Chat Forums > Search Engines > MSN Search Optimization > Robots.txt and msnbot

« Previous Thread | Next Thread »

Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Linear Mode Linear Mode
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump




 Free IT White Papers!
 
Create the Optimal Architecture for your Critical Applications
Warburton's the largest independently owned bakery in the UK faced a number of difficult challenges in providing the most robust yet efficient IT infrastructure for their organization's success. IBM's services combined with their xSeries servers created the perfect platform for their SAP environment with sufficient flexibility, and did so in very time effective fashion.
Request Your Free Technology Downloads!
 
Five Best Practices for Deploying a Successful Service-Oriented Architecture
This white paper describes the benefits you can expect with SOA, and how IBM can help take your business there.
Request Your Free Technology Downloads!
 
Gartner Magic Quadrant for Application Delivery Controllers
Gartner summarizes its view on Application Delivery Controllers, evaluates strengths and weaknesses of solutions, and provides Magic Quadrant reporting for a quick comparison across all vendors. Learn from Gartner how you can benefit from an all-in-one device like Citrix NetScaler that delivers the highest levels of availability, performance and security.
Request Your Free Technology Downloads!
 
Knowledge is Power
What you don't know can hurt you, and is likely costing you money and increasing your security risks during an era of scarce resources. This white paper proposes six key strategies that enterprise security managers can use to improve their network defense posture.
Request Your Free Technology Downloads!
 
Rationalizing the Multi-Tool Environment
The rationalized multi-tool approach is flexible, scalable and cost effective. It provides the necessary input to the IT service management business processes. It preserves prior investments in monitoring tools, empowers technologists to select the best tools with which to do their jobs, and enhances effective response to incidents.
Request Your Free Technology Downloads!
 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 




© 2003-2010 by Developer Shed. All rights reserved. DS Cluster 11 Hosted by Hostway. 
For more Enterprise Application Development news, visit eWeek