Search Engine Spiders
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
 
User Name:
Password:
Remember me
Go Back   SEO Chat ForumsSearch Engine StrategiesSearch Engine Spiders

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread SEO Chat Forums Sponsor:
Application developers can seamlessly integrate the Advantage Database install with their application install. Learn the best practices used when setting up silent installs with this seminar.
  #1  
Old November 29th, 2007, 01:46 PM
Swirleigh's Avatar
Swirleigh Swirleigh is offline
SEO Gawdess
SEO Chat Novice (500 - 999 posts)
 
Join Date: Jul 2006
Location: San Francisco Bay Area
Posts: 514 Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 12 h 25 m 59 sec
Reputation Power: 6
Mod_rewrite & robots.txt

I wish to disallow indexing of a couple directories of a website that is using mode-rewrite.

Do I list the "real" directory folder names or the mode-rewriten names in the robots.txt for exclusion?

I am thinking the mode-rewrite names, but I want to make sure.

Reply With Quote
  #2  
Old November 29th, 2007, 03:03 PM
dzine's Avatar
dzine dzine is offline
Vergruizer: Vot tebe khuy
SEO Chat Intermediate (1500 - 1999 posts)
 
Join Date: Oct 2005
Location: in a life preserver @ seorefugee
Posts: 1,841 dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Month 4 Days 14 h 41 m
Reputation Power: 20
That depends.

If you still have links - either internal or inbound - pointing to the old URLs then you have to block those as well.

For starters, why not try a site:yoursite.com search on google and yahoo to see if they still have some of the "old" urls in the index
__________________
Learn about the Sopranos...Check your gender ...

Reply With Quote
  #3  
Old November 30th, 2007, 12:18 PM
Swirleigh's Avatar
Swirleigh Swirleigh is offline
SEO Gawdess
SEO Chat Novice (500 - 999 posts)
 
Join Date: Jul 2006
Location: San Francisco Bay Area
Posts: 514 Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 12 h 25 m 59 sec
Reputation Power: 6
The site is under developement. There is no "old" urls.

I am kind of new to mode-rewrite... and usually want all sections of site listed, but we are creating a section of site that users can only see after filling in a form. That section I do not want listed in search results. So... that section I want to disallow in robots.txt.

Question being... to disallow the mode-rewrite version name directories (/topic/detail.html) of pages, or... to disallow the actual folders in my ftp list for those pages (/after-form/subject123id549087723ddsw/)?

Reply With Quote
  #4  
Old November 30th, 2007, 12:45 PM
dzine's Avatar
dzine dzine is offline
Vergruizer: Vot tebe khuy
SEO Chat Intermediate (1500 - 1999 posts)
 
Join Date: Oct 2005
Location: in a life preserver @ seorefugee
Posts: 1,841 dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level)dzine User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Month 4 Days 14 h 41 m
Reputation Power: 20
Just disallow the url that you use on your pages

Reply With Quote
  #5  
Old November 30th, 2007, 12:45 PM
JagNet's Avatar
JagNet JagNet is online now
Smoke me a kipper...
Click here for more information
 
Join Date: Aug 2007
Posts: 1,456 JagNet User rank is Sergeant Major (2000 - 5000 Reputation Level)JagNet User rank is Sergeant Major (2000 - 5000 Reputation Level)JagNet User rank is Sergeant Major (2000 - 5000 Reputation Level)JagNet User rank is Sergeant Major (2000 - 5000 Reputation Level)JagNet User rank is Sergeant Major (2000 - 5000 Reputation Level)JagNet User rank is Sergeant Major (2000 - 5000 Reputation Level) 
Time spent in forums: 3 Weeks 6 Days 6 h 40 m 4 sec
Reputation Power: 23
Use the rewritten version (that's the version that Google sees).
__________________
!! Peanut free and Google friendly !!
New to SEO? SEOChat SEO FAQs
Forum Rules and Posting Guidelines
URL canonicalization code solutions

Reply With Quote
  #6  
Old November 30th, 2007, 12:49 PM
Swirleigh's Avatar
Swirleigh Swirleigh is offline
SEO Gawdess
SEO Chat Novice (500 - 999 posts)
 
Join Date: Jul 2006
Location: San Francisco Bay Area
Posts: 514 Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level)Swirleigh User rank is Corporal (100 - 500 Reputation Level) 
Time spent in forums: 6 Days 12 h 25 m 59 sec
Reputation Power: 6
Thanks!

Reply With Quote
Reply

Viewing: SEO Chat ForumsSearch Engine StrategiesSearch Engine Spiders > Mod_rewrite & robots.txt


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

 Free IT White Papers!
 
Accelerating Trading Partner Performance
One in five. That's how many partner transactions have at least one error. That is an amazing statistic, particularly given the extraordinary leaps in innovation across the global supply chain during the past two decades. Download this white paper to learn more.

 
Competing on Analytics
This Tech Analysis is designed to help identify characteristics shared by analytics competitors, and includes information about 32 organizations that have made a commitment to quantitative, fact-based analysis.

 
Cost Effective Scaling with Virtualization and Coyote Point Systems
An overview of the industry trend toward virtualization, how server consolidation has increased the importance of application uptime and the steps being taken to integrate load balancing technology with virtualized servers.

 
Five Checkpoints to Implementing IP Telephony
Implementation planning for IP PBX software and IP telephony has become vital as businesses replace discontinued legacy PBX phone systems. This informative whitepaper outlines five "checkpoints" for any implementation plan that will help make IP communications a successful proposition.

 
Hosted Email Security: Staying Ahead of New Threats
In the last two years, email has become a fierce battleground between the nefarious forces of spam and malware, and the heroes of messaging protection. The spam volumes increased alarmingly every month, bringing clever new forms of phishing and virus propagation attacks.

 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 2 hosted by Hostway