
August 11th, 2008, 05:19 PM
|
|
Contributing User
|
|
Join Date: Mar 2003
Location: London UK
Posts: 156
 
Time spent in forums: 2 Days 21 h 39 m 20 sec
Reputation Power: 6
|
|
|
Removing 100's of urls
Hello,
I have quite a few pages within my web site which I'd like google to stop indexing. This has happened mainly due to products which no longer exist, hence I then delete the product from the database.
These URLs only exist because I cannot make it redirect a 301 to the homepage(ideally) or 404, it just generates the standard header.php left and right sections of the web site along with a blank page in the mid section where the product detail used to exist.
Because of a url rewrite rule we use, any url you type into the browser such as thedomainname.com/real-product.html and thedomainname.com/anything-here-at-all.html will load a page. The latter being the problem! I've asked the web designer to find a way of only allowing actual URLs from products that exist in the database, but they haven't found a solution for this just yet, albeit it's only a few days since I asked them, but they don't seem confident ATM.
Whats the best way of having google delete these products from their index without using any 301 or internal fixes?
Is it possible to disallow a whole list of specific URLs using a robots.txt file?
Thanks
__________________
UK Supplier of Glow Sticks and other glow products.
|