MSN Search Optimization
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
 
User Name:
Password:
Remember me
Go Back   SEO Chat Forums > Search Engines > MSN Search Optimization
Reload this Page

MSNbot is crawling our sites everyday.

Discuss MSNbot is crawling our sites everyday. in the MSN Search Optimization forum on SEO Chat.
MSNbot is crawling our sites everyday. MSN Search Optimization forum discussing the Microsoft search engine for MSN and Live.com. Find tips to help you get high rankings in Microsoft's search. Discuss what factors are most important to its algorithm.

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Page 1 of 2 1 2 >
« Previous Thread | Next Thread »  
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
  #1  
Old May 6th, 2004, 02:30 PM
sorvoja's Avatar
sorvoja sorvoja is offline
Contributing User
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Apr 2004
Posts: 1,547 sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 6 Days 1 h 11 m 41 sec
Reputation Power: 11
Question MSNbot is crawling our sites everyday.
Hi,

I see that MSN really has picked up the pace with their beta spider. Most of the pages on the web sites we manage are crawled everyday. I would estimate that MSN is four or five times as active as Google.

Do anyone else have any comments or insights on this.
I wonder how large the MSN index will be when it becomes available trough a search interface.

Any and all comments are highly appreciated. :-)
Reply With Quote
  #2  
Old May 6th, 2004, 04:09 PM
pk_synths's Avatar
pk_synths pk_synths is offline
Contributing User
SEO Chat Specialist (4000 - 4499 posts)
  
Join Date: May 2003
Location: Chicago
Posts: 4,109 pk_synths User rank is Corporal (100 - 500 Reputation Level)pk_synths User rank is Corporal (100 - 500 Reputation Level)pk_synths User rank is Corporal (100 - 500 Reputation Level)pk_synths User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 5 Days 17 h 45 m 3 sec
Reputation Power: 12
Send a message via AIM to pk_synths Send a message via Yahoo to pk_synths
Noticed the same thing. MSNBots on crack! Never thought I would see the days that MSNBot was more active than Googlebot.

Now to only slip Yahoo something.
__________________
-PK

Litigation Support
Reply With Quote
  #3  
Old May 6th, 2004, 10:28 PM
dejaone dejaone is offline
Contributing User
SEO Chat Beginner (1000 - 1499 posts)
  
Join Date: Apr 2004
Posts: 1,178 dejaone User rank is Lance Corporal (50 - 100 Reputation Level)dejaone User rank is Lance Corporal (50 - 100 Reputation Level)dejaone User rank is Lance Corporal (50 - 100 Reputation Level) 
Time spent in forums: 6 Days 23 h 17 m 45 sec
Reputation Power: 6
all big three crawl my site everyday. But only Google generates lot of traffic for me.
Reply With Quote
  #4  
Old May 6th, 2004, 11:15 PM
jfontestad's Avatar
jfontestad jfontestad is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Apr 2004
Location: New Orleans, La.
Posts: 388 jfontestad User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 6 h 24 m 8 sec
Reputation Power: 5
Send a message via AIM to jfontestad
Hey,
MSN has been crawling my sites lately also. I'd say it has
crawled about 70,000+ and it's still crawling. Should I let it keep crawling?
or is it just waisting my bandwidth ? Another thing I'm working on
changing my URL scheme giving it a new look, should I keep the old URL
scheme + just implement the new one or should I get rid of the old one
and just go with the new one? All my indexed pages have the old URL
format, what would you guys/girls suggest?
__________________
J Fontestad
-PreMade MySpace Layouts
-PS2 Cheat Codes-PSP Cheat Codes
-WMtoolkit providing you with the best SEO Tools
Reply With Quote
  #5  
Old May 6th, 2004, 11:40 PM
sorvoja's Avatar
sorvoja sorvoja is offline
Contributing User
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Apr 2004
Posts: 1,547 sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 6 Days 1 h 11 m 41 sec
Reputation Power: 11
Hi,

I do not know. I would not be comfortable blocking MSN for any reason.
Reply With Quote
  #6  
Old May 7th, 2004, 12:59 AM
pk_synths's Avatar
pk_synths pk_synths is offline
Contributing User
SEO Chat Specialist (4000 - 4499 posts)
  
Join Date: May 2003
Location: Chicago
Posts: 4,109 pk_synths User rank is Corporal (100 - 500 Reputation Level)pk_synths User rank is Corporal (100 - 500 Reputation Level)pk_synths User rank is Corporal (100 - 500 Reputation Level)pk_synths User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 5 Days 17 h 45 m 3 sec
Reputation Power: 12
Send a message via AIM to pk_synths Send a message via Yahoo to pk_synths
Quote:
Should I let it keep crawling? or is it just waisting my bandwidth?


Whats bandwidth???

Honestly that was going to be my reply but I know alot of people will be goin' "You dont know what bandwidth is?!!?" and junk like that so the point is:

Is MSN really eating up that much bandwidth that you'll exceed any insane number of GBs your host gives you? There. Happy?

Quote:
Another thing I'm working on changing my URL scheme giving it a new look, should I keep the old URL scheme + just implement the new one or should I get rid of the old one and just go with the new one?


301 redirect all your old pages to your new ones to keep backlinks and value of those pages.
Reply With Quote
  #7  
Old May 7th, 2004, 01:49 AM
jfontestad's Avatar
jfontestad jfontestad is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Apr 2004
Location: New Orleans, La.
Posts: 388 jfontestad User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 6 h 24 m 8 sec
Reputation Power: 5
Send a message via AIM to jfontestad
that's a lot of pages though. maybe i can figure something out with mod_rewrite
but if anyone knows how i can do this please pm me. i would very much appreciate
it.
thank you.
Reply With Quote
  #8  
Old May 7th, 2004, 04:40 PM
sorvoja's Avatar
sorvoja sorvoja is offline
Contributing User
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Apr 2004
Posts: 1,547 sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 6 Days 1 h 11 m 41 sec
Reputation Power: 11
MSNbot is still going strong.

I wonder if they intend to crawl the entire web every 24 hours?

This is nuts.
Reply With Quote
  #9  
Old May 9th, 2004, 11:47 AM
sorvoja's Avatar
sorvoja sorvoja is offline
Contributing User
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Apr 2004
Posts: 1,547 sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 6 Days 1 h 11 m 41 sec
Reputation Power: 11
MSN is still crawling everyday!
Reply With Quote
  #10  
Old May 9th, 2004, 12:18 PM
Cygnus's Avatar
Cygnus Cygnus is offline
Wine Geek
SEO Chat Beginner (1000 - 1499 posts)
  
Join Date: Oct 2003
Location: Cave Creek, AZ
Posts: 1,304 Cygnus User rank is Corporal (100 - 500 Reputation Level)Cygnus User rank is Corporal (100 - 500 Reputation Level)Cygnus User rank is Corporal (100 - 500 Reputation Level)Cygnus User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 4 h 19 m 2 sec
Reputation Power: 9
I wonder if it might be stuck.
__________________
Do you really need a successful link building campaign? Then you absolutely must use these guys: Free links from Digitalpoint's CO-OP & Free links from Link Vault
Reply With Quote
  #11  
Old May 9th, 2004, 12:19 PM
Xia Xia is offline
Contributing User
SEO Chat Newbie (0 - 499 posts)
  
Join Date: Nov 2003
Posts: 206 Xia User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 6
Send a message via MSN to Xia
Is there anywhere we can go to check the BETA search results?
Reply With Quote
  #12  
Old May 9th, 2004, 01:37 PM
sorvoja's Avatar
sorvoja sorvoja is offline
Contributing User
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Apr 2004
Posts: 1,547 sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 6 Days 1 h 11 m 41 sec
Reputation Power: 11
Hi Xia,

No, I do not think so.
Reply With Quote
  #13  
Old May 9th, 2004, 04:24 PM
Jocelyn's Avatar
Jocelyn Jocelyn is offline
Contributing
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Sep 2003
Location: Montreal / Canada
Posts: 1,658 Jocelyn User rank is Corporal (100 - 500 Reputation Level)Jocelyn User rank is Corporal (100 - 500 Reputation Level)Jocelyn User rank is Corporal (100 - 500 Reputation Level)Jocelyn User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 1 Week 3 Days 4 h 21 m 4 sec
Reputation Power: 11
Quote:
Originally Posted by Cygnus
I wonder if it might be stuck.

It's activity started to increase last month according to my logs.
Reply With Quote
  #14  
Old May 10th, 2004, 05:19 AM
sorvoja's Avatar
sorvoja sorvoja is offline
Contributing User
SEO Chat Intermediate (1500 - 1999 posts)
  
Join Date: Apr 2004
Posts: 1,547 sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level)sorvoja User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 6 Days 1 h 11 m 41 sec
Reputation Power: 11
Hi,

MSN has started todays crawl.
Reply With Quote
  #15  
Old May 10th, 2004, 06:30 AM
SEO_AM's Avatar
SEO_AM SEO_AM is offline
B afraid.. B very afraid!
Click here for more information.
  
Join Date: Mar 2004
Location: Land of enchantment... deserts of the Southwest
Posts: 6,229 SEO_AM User rank is Second Lieutenant (5000 - 10000 Reputation Level)SEO_AM User rank is Second Lieutenant (5000 - 10000 Reputation Level)SEO_AM User rank is Second Lieutenant (5000 - 10000 Reputation Level)SEO_AM User rank is Second Lieutenant (5000 - 10000 Reputation Level)SEO_AM User rank is Second Lieutenant (5000 - 10000 Reputation Level)SEO_AM User rank is Second Lieutenant (5000 - 10000 Reputation Level)SEO_AM User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 2 Months 3 Weeks 4 Days 19 h 26 m 50 sec
Reputation Power: 61
... Until MSN starts giving me some numbers in the referrals area I think all this talk is interesting, but immaterial. Once they start using their "new" relevant serps maybe I will get excited. Want them to crawl the hell out of my site, but I am into results.
Reply With Quote