#1
  1. No Profile Picture
    Registered User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Mar 2014
    Posts
    20
    Rep Power
    0

    How to Fix Crawl Errors


    On my WMT (now Search Console) I have about 1,000 crawl errors. But when I copy the url and search for it in the sitemap they are nowhere in there. How is it a crawl error if those urls are not on the sitemap to tell Google to crawl them in the first place?
  2. #2
  3. My views are my own ;)
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    Jun 2012
    Location
    Clifton Park, New York
    Posts
    1,217
    Rep Power
    1739
    Obviously Google is somehow able to get there, so those pages are linked to from your other pages or external pages. You need to find the pattern and understand how Google is getting there first.

    Comments on this post

    • Will-O-The-Wisp agrees
    Everything will be ok in the end

    If it's not ok, it's not the end
  4. #3
  5. Super Moderator
    SEO Chat Good Citizen (1000 - 1499 posts)

    Join Date
    May 2015
    Location
    United Kingdom
    Posts
    1,393
    Rep Power
    2207
    Originally Posted by use3
    On my WMT (now Search Console) I have about 1,000 crawl errors. But when I copy the url and search for it in the sitemap they are nowhere in there. How is it a crawl error if those urls are not on the sitemap to tell Google to crawl them in the first place?
    Google doesn't use the sitemap to crawl, it uses it to help you crawl, but a crawl is it literally just clicking through every link in the site.

    So use a software like ScreamingFrog to find out where these pages are linked in your site.
    Always happy to help in private message. | Learn about my latest project Can You Get Automated Pages To Rank In Google?
    Useful Links: Useful SEO Tools | GTmetrix | FileOptimizer | Cloudflare
  6. #4
  7. SEO Since 97
    SEO Chat Mastermind (5000+ posts)

    Join Date
    Mar 2011
    Location
    Arizona
    Posts
    7,686
    Rep Power
    4753
    Do you delete an auto once it's been sold?
  8. #5
  9. No Profile Picture
    Contributing User
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Aug 2015
    Posts
    22
    Rep Power
    0
    Use3-

    I had a similar issue with a website I built for a moving company. We'd notice intermittent crawl errors and I had no idea why as ALL the links throughout the website checked out OK. The website was hosting on a Rackspace dedicated server. I had a hunch - moved it to a cloud server - and presto... no more crawl errors.

    It could be your solution, albeit a long shot.

    - KG
  10. #6
  11. No Profile Picture
    Newbie
    SEO Chat Explorer (0 - 99 posts)

    Join Date
    Aug 2015
    Posts
    1
    Rep Power
    0
    Errors in sitemaps are often caused by old sitemaps that have since 404’d, or pages listed in the current sitemap that return a 404 error. Make sure that all the links in your sitemap are quality working links that you want Google to crawl. Rather than create a temporary fix, its better to fix 404 errors permanently and Google Webmaster Tools provides one of the best solutions.
  12. #7
  13. Founder, SEOChat.com :)
    SEO Chat Skiller (1500 - 1999 posts)

    Join Date
    Dec 2002
    Location
    Florida, Madrid, Berlin, Dublin
    Posts
    1,820
    Rep Power
    525
    Originally Posted by HomeButton
    Use3-
    moved it to a cloud server - and presto... no more crawl errors.

    It could be your solution, albeit a long shot.

    - KG
    I highly doubt this is related. Interesting coincidence though. If Google is requesting pages from a domain, then the server it's hosted on should be irrelevant. The only situation I can think of where this might be the case is if there are other domains hosted on the server, or pages are being requested directly from the IP, and those requests are being re-written or otherwise translated to requests on the domain experiencing the problems.

    More likely here is that there are links on third-party sites to pages that no longer exist (as mentioned by Ann above).

Similar Threads

  1. Crawl Errors: 2 errors/ 2 html pages showing.
    By Masseffect3 in forum Google Optimization
    Replies: 5
    Last Post: Sep 12th, 2013, 07:03 PM
  2. Crawl Errors
    By sebcomkt in forum Google Optimization
    Replies: 6
    Last Post: Mar 10th, 2013, 08:51 AM
  3. GWT & Crawl Errors!
    By rizmriz in forum Google Optimization
    Replies: 1
    Last Post: Apr 20th, 2012, 08:41 AM
  4. Crawl errors
    By agmccall in forum Search Engine Optimization
    Replies: 4
    Last Post: Jul 29th, 2011, 05:09 AM
  5. GWT - Crawl Errors
    By Hilary in forum Google Optimization
    Replies: 1
    Last Post: Mar 6th, 2009, 09:14 AM

IMN logo majestic logo threadwatch logo seochat tools logo