|
|||||||||
|
|||||||||
|
|||||||||
 |
|||||||||
| |
||
|
|
|
|
|||||||||
|
|||||||||
|
|||||||||
|
|||||||||
| |
||
|
|
|
| |||||||||
 |
|
|
«
Previous Thread
|
Next Thread
»
|
Thread Tools | Search this Thread | Rate Thread | Display Modes |
SEO Chat Forums Sponsor:
|
|
|
|
#1
April 20th, 2004, 12:23 PM
|
|||
|
|||
|
robots and google
1 User-agent: Mediapartners-Google* 2 Disallow: This change will allow us to crawl your site and deliver targeted ads to your users. 3 User-agent: * 4 Disallow: /html/ 5 Disallow: /logs/
Can someone explain 2? This is my robots.txt file and I'm not sure what it means. Should I remove 2 if I'm serving Adsense? I just don't want spiders in my logs and /html/ directory.
|
|
#2
April 20th, 2004, 12:49 PM
|
||||
|
||||
|
Number 1 will allow Adsense to spider your site so that it can display relevant ads. EDIT: Make sure any files that you want Adsense to spider are NOT in the HTML folder.
Number 2 prevents all spiders from crawling and indexing any file that is in your HTML folder and logs directory. Just make sure there aren't any files inside the HTML folder that you would like to include in the index. Adsense, I dont think, cares if the page the ads are displayed on is in the index or not.
|
|
| Viewing: SEO Chat Forums > Other > HTML Coding > robots and google |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|
|
 |
|
|
|
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
