Spiders not visiting

Hi

I'm new here so if this thread breaks a forum rule I missed then I apologise, just remove the problem bits.

I have developed a number of websites and am normally pretty successful at getting them up in the search engine rankings. However there is one site which I can't even get to be visited by spiders, with the exception of an occasional visit from Google. It has been up for a year now and is still not getting referrals.

I have sitemaps (html and xml), it has been submitted to the search engines, there is a robots.txt file and I use the robots tag. I can only assume there is something about the site which is causing the problem but I can't see what.

I can't post a link yet but if anybody is interested enough to take a look at the site please PM me and I will send you the URL.

Any guidance would be much appreciated.

Ian

edit - I hope somebody spots this. The thread has been moved to a section in which I am not allowed to post new replies. Any chance it could be moved back?

The URL of the problem site is ppbb dot uk dot com

Thanks

You can post the site URL.
Remember it should not be live.
http://www.yoursite.com/, post it in this way.

Welcome to SEOchat forums Ian

You can post the URL in domain(dot)com format for others to see and review it.

Have a pleasant stay here.

Moving this thread to the Search Engine Spiders forum.

How do you want to get a site in the search engines when there is no external link pointing to it ?

Jean-Luc

I agree it won't get a high ranking with no external links, but the problem at the moment is that it is not even being visited by spiders. The links are being worked on.

Thanks for moving the thread by the way, nice to be able to post on it again. :0)

Your site is being visited by spiders,google last cache date is 22nd oct'2007.
I think your main concern is that your site is not getting good rank.
Try to get some back links from relevant sites and use your targeted key words or phrases as anchor text.
Hope you will get what you want.

I think the site has some back links and thats enough to get in to the SE.The site is being visited by google spiders.
Here the main concern is the site is not getting rank in targeted keywords.

No, sorry to disagree but my concern is not the ranking in Google. I am concerned that the site is only being visited by Google spiders and no others. Google has only just started visiting it in the past few weeks after I resubmitted the site, not sure why it took this time and not before but it still doesn't appear to have taken with the other engines.

Build links to your site and in addition to that fix the following issues:

* Canonical Issue: You have a canonical issue with your site. There are multiple versions of your site such as http://www.ppbb.uk.com/ ; http://ppbb.uk.com/ ; http://www.ppbb.uk.com/index.html and http://ppbb.uk.com/index.html

Now the search engine might take up any one of these URLs and hence you might not find your site when you search with another URL. Canonical issues may also raise some duplicate content complications for your site.

I found that you are running your site on Apache, hence put the following codes in your .htaccess file and upload it to your domain's root folder.

Make sure that you have the following at the beginning of your .htaccess file



For redirecting non-www to www


For redirecting /index.html to /



Hope this helps

Thanks, I'll look into that. I've just checked the other domains I am supporting and they are all the same, so will look into those also. However, they are all getting good rankings so I would be surprised if that is the main problem (unless I have just been lucky before).

I have asked my host to check if there is anything they should be doing, as I can't believe it should need all that in the .htaccess file. After all a large percentage of sites are probably developed by people who don't even know about that file.