|
|
|||||||||
|
|||||||||
|
|||||||||
| |
||
| |||||||||
![]() |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
|
|
Application developers can seamlessly integrate the Advantage Database install with their application install. Learn the best practices used when setting up silent installs with this seminar. |
|
#1
|
||||
|
||||
|
Mod_rewrite & robots.txt
I wish to disallow indexing of a couple directories of a website that is using mode-rewrite.
Do I list the "real" directory folder names or the mode-rewriten names in the robots.txt for exclusion? I am thinking the mode-rewrite names, but I want to make sure. |
|
#2
|
||||
|
||||
|
That depends.
If you still have links - either internal or inbound - pointing to the old URLs then you have to block those as well. For starters, why not try a site:yoursite.com search on google and yahoo to see if they still have some of the "old" urls in the index
__________________
|
|
#3
|
||||
|
||||
|
The site is under developement. There is no "old" urls.
I am kind of new to mode-rewrite... and usually want all sections of site listed, but we are creating a section of site that users can only see after filling in a form. That section I do not want listed in search results. So... that section I want to disallow in robots.txt. Question being... to disallow the mode-rewrite version name directories (/topic/detail.html) of pages, or... to disallow the actual folders in my ftp list for those pages (/after-form/subject123id549087723ddsw/)? |
|
#4
|
||||
|
||||
|
Just disallow the url that you use on your pages
|
|
#5
|
||||
|
||||
|
Use the rewritten version (that's the version that Google sees).
__________________
!! Peanut free and Google friendly !!
New to SEO? SEOChat SEO FAQsForum Rules and Posting Guidelines URL canonicalization code solutions |
|
#6
|
||||
|
||||
|
Thanks!
|
![]() |
| Viewing: SEO Chat Forums > Search Engine Strategies > Search Engine Spiders > Mod_rewrite & robots.txt |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
|
|
|
|
|