#1
  1. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2006
    Posts
    34
    Rep Power
    13

    Does the sitemap xml file have to be linked to get crawled?


    hi,

    well i have a sitemap.xml file in my website, and google knows where it is because i've added it in the google sitemaps webpage.
    do i have to make a link to it from my homepage for every search engine bots to crawl it or they will be able to spider it even if there is no link leading to it?
  2. #2
  3. rod@missionop.com
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Mar 2003
    Location
    Palm Beach Gardens FL 33410
    Posts
    16,980
    Rep Power
    0
    Originally Posted by paokun
    hi,

    well i have a sitemap.xml file in my website, and google knows where it is because i've added it in the google sitemaps webpage.
    do i have to make a link to it from my homepage for every search engine bots to crawl it or they will be able to spider it even if there is no link leading to it?
    Nope - no need - just update it when you add new pages.
  4. #3
  5. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2006
    Posts
    22
    Rep Power
    0
    Originally Posted by paokun
    hi,

    well i have a sitemap.xml file in my website, and google knows where it is because i've added it in the google sitemaps webpage.
    do i have to make a link to it from my homepage for every search engine bots to crawl it or they will be able to spider it even if there is no link leading to it?
    yes and no. with google youve already submitted the sitemap to them, so they have it. however, we may start to see other (lesser known) search engines and spiders start to adapt sitemap like files. because of this i have an html sitemap with a link in saying XML version here, which links to sitemap.xml.

    ryan
  6. #4
  7. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2005
    Posts
    14
    Rep Power
    0
    I have sitemap.xml.gz , is it the same as sitemap.xml ? (I mean it's only compressed, but can google and other search engines' spiders read it, as it is gzipped?)
  8. #5
  9. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2006
    Posts
    34
    Rep Power
    13
    hi,
    well i've linked it just in case..even if someone clicks on it it will be just an xml file. thanks for your replies:-)
  10. #6
  11. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Jan 2005
    Posts
    70
    Rep Power
    14
    google can read gzipped, im pretty sure of it. I don't know of any search engines which use xml map format other than Google.
  12. #7
  13. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2005
    Posts
    14
    Rep Power
    0
    Google sets standards, so I think other SEs will start accepting Google formatted sitemaps very soon....
  14. #8
  15. My first avatar here
    SEO Chat Skiller (1500 - 1999 posts)

    Join Date
    Jun 2005
    Location
    San Jose, Costa Rica
    Posts
    1,928
    Rep Power
    22
    i know yahoo can read xml, becuase their sitemaps are xml files
    I am a content scrapper. If you need to obtain and process big data, pm me.
    I have a multi-threaded spider that runs on several ec2 instances.
    I have been crawling twitter, and other big guys for a while. Outputs: SQL, XML, CSV, MYSQL, TXT
  16. #9
  17. SEO at Sea
    SEO Chat Discoverer (100 - 499 posts)

    Join Date
    Apr 2006
    Location
    Utah
    Posts
    331
    Rep Power
    17

    you'd think


    Originally Posted by manca
    I have sitemap.xml.gz , is it the same as sitemap.xml ? (I mean it's only compressed, but can google and other search engines' spiders read it, as it is gzipped?)
    You would think so, but I had to unzip my .gz file for google to finally see and read it in sitemaps. Maybe it was a fluke, or user error (HA, yea right)..but, to elimnate further problems, I always unzip mine just in case.
  18. #10
  19. Moderator
    SEO Chat Scholar (3000 - 3499 posts)

    Join Date
    Sep 2005
    Location
    NY
    Posts
    3,410
    Rep Power
    161
    Originally Posted by ryant
    yes and no. with google youve already submitted the sitemap to them, so they have it. however, we may start to see other (lesser known) search engines and spiders start to adapt sitemap like files. because of this i have an html sitemap with a link in saying XML version here, which links to sitemap.xml.

    ryan
    That is probably the best way to go. I use a html sitemap for that reason and a xml sitemap which I use for Google however MSN and Yahoo seem to crawl my HTML one fine. Unless you have 1000's of pages I think HTML sitemaps work really well for most search engines.
  20. #11
  21. Super Moderator
    SEO Chat Super Hero (2500 - 2999 posts)

    Join Date
    Aug 2003
    Location
    Vancouver, Washington, U.S.A.
    Posts
    2,578
    Rep Power
    669
    No you dont need to link it from any page of your site, as long as you hav it in google site maps. Its not a bad idea to have it linked however for reasons discussed.-
    Last edited by googler; May 23rd, 2006 at 12:41 PM.
    Work Smarter, Not Harder
  22. #12
  23. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    May 2006
    Posts
    34
    Rep Power
    13
    thanks everyone:-) i've linked it, in any case it wouldnt hurt anyone to have one more link
  24. #13
  25. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2005
    Posts
    14
    Rep Power
    0
    Originally Posted by googler
    google site-maps uses .xml only. No compressed or zipped files. Yahoo reads xml site-maps.
    No you dont need to link it from any page of your site, as long as you hav it in google site maps. Its not a bad idea to have it linked however for reasons discussed.
    Are you sure google can't read compressed file? As in their manual is the example which shows to submit "sitemap.xml.gz" to their sitemap service....I think they(the SEs) can still read .gz files...
  26. #14
  27. Super Moderator
    SEO Chat Super Hero (2500 - 2999 posts)

    Join Date
    Aug 2003
    Location
    Vancouver, Washington, U.S.A.
    Posts
    2,578
    Rep Power
    669
    Originally Posted by manca
    Are you sure google can't read compressed file? As in their manual is the example which shows to submit "sitemap.xml.gz" to their sitemap service....I think they(the SEs) can still read .gz files...
    I was wrong I stand corrected
    from google:
    Can I zip my Sitemaps or do they have to be gzipped?

    Please use gzip to compress your Sitemaps.


    Google also accepts rss and atom feeds and a couple of other protocols

Similar Threads

  1. SSI: Why and How to Use Server Side Includes
    By amabaie in forum SEO Chat Articles
    Replies: 0
    Last Post: May 20th, 2004, 10:07 PM
  2. strange log
    By zygimantas in forum Google Optimization
    Replies: 1
    Last Post: Dec 15th, 2003, 11:15 PM

IMN logo majestic logo threadwatch logo seochat tools logo