Building Non-Crawlable Links w/Javascript

I've searched the archive and read through a few of the threads about javascript and I haven't found a good solution to creating non-crawlable links.

The problem:
My site needs a lot of links as the content can be sliced many different ways and I want our customers to easily find what they're looking for. However.... Google sees all the links and devalues all the pages.

The solution:
Make 50% of the links some other way so people can still click them but the search engines don't see them. I currently do this through a document.write in a separate .js file which I include.

I later wrote my own page crawler to analyse my site which uses the webbrowser object, and the webbrowser object sees ALL of the links whether they're in the javascript or not.

I'm thinking if I could put the links in a function and invoke the function on page load that maybe the webbrowser object (and the search engine crawler) wouldn't see the links but our customers would.

Any ideas would be appreciated...

Thanks.
Chris.

No need to make it complicated. Simply add a rel="nofollow" to the link code.

raz

Not only that, do you really want to find yourself under the microscope for having "apparently" hidden half of your links from the SE's. How long before someone reports that one?

If it were all of the links, or just a menu...Then no worries, but selective Java scripting to prevent disclosing links from Google cannot be a good thing.

And...as stated above, its far more complicated than necessary, or proper.

--Melanie

Sorry Melanie, but I don't quite see that as being against TOS or in any way unethical. Hiding links/content from search engines is a perfectly normal thing to do (think robots.txt or DHTML) and Google has never said that you either have to show all your links to them or none at all. What would someone report you for?

O don't know seems lately you'll get a report for looking sideways.

http://www.google.com/support/webmasters/bin/answer.py?answer=66353&query=hiding+links&topic=&type=



It's all in the interpretation.

http://www.google.com/support/webmasters/bin/answer.py?answer=66355&query=pages+for+search+engines&topic=&type=





As I said, under the microscope, as just choosing half of the links to be in Java script would be quite unnatural. Google is receiving a ton of reports...Webmasters are using the form to turn each other in for crazy stuff, and even if it's NOT a clear violation, it has to be checked out by Google.

So something like this, and Google comes calling...I am seeing in many cases...They find something small and in consequential on the original report. However, they is generally something else in relative plain view to add to the case and wrap it up upon investigation. Maybe in addition to something like this, you also have CSS display none hidden text, bingo...Penalty.

My point here is NOT to draw attention with these types of actions, as many web sites are going to have "other" issues as well.

--Melanie

If you worry about pages being devalued, you should add a rel="nofollow" to the links as others said. If you seriously don't want the other pages indexed at all, you also need to exclude them via the robots.txt or with the meta-robots element.

ok, I see where you're coming from here, but:


Google is not saying you must make the Javascript content available to non-javascript enabled browsers & crawlers, but if you do then make sure that it's the same and don't use the <noscript> tags to show alternative content.



Again, the recommendation is to consider providing Javascript content in a <noscript> tag for the purposes of accessibility, and if you do then "ensure that you provide the same content in both elements".


The intent here is along similar lines to Google's own recommendations re the use of the nofollow attribute: that you can legitimately use javascript or nofollow in order to not pass value to the linked page.

Hiding links from the user but not the search engines is against TOS - hiding links from the search engines but not the user is not.


Regarding the use of the spam reporting tool, Google does not have to follow up every report. In fact they don't. It's not scalable.
Only when something repeatedly shows up in reports do they look into it, and then prefer to deal with it algorithmically if they can. Only in blatant cases where an algorithmic solution is not appropriate would they turn to manual intervention.


There are perfectly legitimate reasons for this - for example for text that is revealed using DHTML (though personally I prefer to set the display to none using javascript for accessibility reasons). But as long as there is some on page method whereby the user can see the content, then it's not hidden. Display:none won't automatically be penalised.