Discuss Crawler shows 404 for existing pages in the Google Optimization forum on SEO Chat. Crawler shows 404 for existing pages Google Optimization forum discussing techniques to increase rank in Google. Find help for defeating spam pages, building site trust, and beating the Google algorithm. Also read about the best SEO tools to help with Google search.
BuySellLinks.com is a new Text Link Advertising Brokerage by the founder of LinkAdage that offers high-end text links and a premier membership base. We screen all advertisers and publishers and their web pages to make sure we are offering a service that works for everyone. Our plug-in is super easy to install but if you are hands off, w e offer free installation.
Tired of dealing with shady text link buyers and/or sellers? Try BuySellLinks.
Receive the tools necessary to be the rock star of your field. Our 12-month program teaches you the evolving world of multi-channel marketing as well as the complex issues and opportunities found in the industry.
Posts: 2
Time spent in forums: 24 m 50 sec
Reputation Power: 0
Crawler shows 404 for existing pages
Hello,
I use DotNetNuke for my web site but i have a strange problem. All the pages that are created as DNN pages are not indexed by Google although they are existing pages. Interestingly Live Search (MSN) have them indexed but Yahoo don't. The error I see in Google's webmaster tools is 404 (Not Found) but when I click on the link I am able to load the page.
Does somebody have any idea why Google is not indexing them?
Here is a link to my sitemap
(URL address blocked: See forum rules)
and the pages that are not indexed are for example the Search Properties, FAQs and the articles.
Posts: 2
Time spent in forums: 24 m 50 sec
Reputation Power: 0
Quote:
Originally Posted by jperks1985
I get this same problem sometimes w/ google 404 not found on 120 links. In my case I believe its a bandwidth issue.
Hmm... I thought the same or eventually because they are marked as spam for whatever reason. My problem is that I don't have a lot of traffic right now and it seems that the application needs some time to "boot" before it starts serving the pages faster and if Google comes once in a while and crawls only one page then I won't have much success.
Posts: 322
Time spent in forums: 3 Days 13 h 59 m 10 sec
Reputation Power: 6
Quote:
Originally Posted by travelweaver
Hmm... I thought the same or eventually because they are marked as spam for whatever reason. My problem is that I don't have a lot of traffic right now and it seems that the application needs some time to "boot" before it starts serving the pages faster and if Google comes once in a while and crawls only one page then I won't have much success.
Thanks!
tw
Yeah, it seems like google can overwhelm my server.
Posts: 1,078
Time spent in forums: 2 Weeks 5 Days 5 h 8 m 52 sec
Reputation Power: 23
If you suspect that Google demands too much from your server, you can always reduce the crawling speed in the Webmaster Tools.
On my sites, there are frequently crawling errors concerning pages that are there after all. But with tens of thousands or more pages and the occasional server instabilities, I'm not surprised if Google misses a page once in a while. I trust that the bot comes back later and is happy to rediscover the page ;-)
Posts: 2,082
Time spent in forums: 1 Month 6 Days 16 h 59 m 42 sec
Reputation Power: 385
If you think that your sites are having problems "booting" up as you call it after having a period of idle time then here is what you should do:
write or setup a spider that calls each page on a frequency just less than the cache timeout period..
so say you have a Java web app, and it is set on the standard cache timeout period of 20 minutes, then just have your spider hit every page every 19 minutes (i'd stagger the hits out so that you hit 1 in 19 pages every minute). This way all your site is always cached, so when googlebot drops by your site is nice and speedy.
Posts: 530
Time spent in forums: 1 Week 10 h 43 m 16 sec
Reputation Power: 33
Quote:
Originally Posted by travelweaver
The error I see in Google's webmaster tools is 404 (Not Found) but when I click on the link I am able to load the page.
Hi,
When the answer from the web server comes very slowly, you never get a 404 code. The 404 code is sent when the web server has determined that the URL you asked for does not exist.
If the pages really exist, I would move to a better host or a better CMS.
Jean-Luc
__________________ AWStats Support : add-on's, extra sections, forum, installation assistance + Professional AWStats Services 200ok.eu Broken Link Checkerfinds 404 errors, error pages with 200 ok status, missing images, protocol errors, password protected pages, bad domain names, redirect loops, parking pages and more
Posts: 167
Time spent in forums: 2 Days 19 h 2 m 23 sec
Reputation Power: 6
Quote:
Originally Posted by travelweaver
Hello,
I use DotNetNuke for my web site but i have a strange problem. All the pages that are created as DNN pages are not indexed by Google although they are existing pages. Interestingly Live Search (MSN) have them indexed but Yahoo don't. The error I see in Google's webmaster tools is 404 (Not Found) but when I click on the link I am able to load the page.
Does somebody have any idea why Google is not indexing them?
Here is a link to my sitemap
(URL address blocked: See forum rules)
and the pages that are not indexed are for example the Search Properties, FAQs and the articles.
Thanks
tw
I think there is a problem of your hosting. If there is request timed out from your server when googlebot comes for crawling site at that time googlebot not found your site and mention as a 404 Not found. so your site is not crawled properly by the googlebot.
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
Linear Mode
