|
|||||||||
|
|||||||||
|
|||||||||
 |
|||||||||
| |
||
|
|
|
|
|||||||||
|
|||||||||
|
|||||||||
|
|||||||||
| |
||
|
|
|
| |||||||||
 |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
SEO Chat Forums Sponsor:
|
|
|
|
#1
November 27th, 2003, 08:02 PM
|
|||
|
|||
|
Disallow Search Engine Indexing
Hi ...
I got a lot of pages listed in the root directory which I WANT search engine to crawl. In each page, I got SSI pointing to other files. What I want is ... those files in my root directory able to include it but I dont want search engine to spider all my ssi files. May I know how I can exclude it from being index ? Assume that my ssi files are stored in /ssi ? Thanks.
|
|
#2
December 4th, 2003, 10:12 PM
|
||||
|
||||
|
Simple enough, put links to each of the pages on your homepage, and the search engine will find them. Or build a sitemap with all your pages (101K Max) and post a link to it. Or get a link from another site point to your links.
Then put all SSI files in a /ssi directory. Make sure everything works correctly too. Create a Robots.txt file like the following example: User-agent: * Disallow: /ssi/ Google will not index files contained in the ssi directory and all your pages will get spidered by Google. 
|
|
| Viewing: SEO Chat Forums > Search Engine Strategies > Search Engine Optimization > Disallow Search Engine Indexing |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
 |
|
|
|
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
