Local Popularity vs. Global Popularity

<removed>

Very interesting and illuminating, Rand. Nice description of these two methodologies.

One problem. Ask Jeeves has a teeny market share. They do interesting stuff, but users don't flock to the engine. On top of that Jeeves still floods its pages with sponsored listings on top. That makes it like reading a newspaper for the ads, watching tv for the ads or putting all your faith in paid infocommercials. The relevancy factor is hidden from the user. In fact the "beauty" of the Teoma technology, that doesn't need to deal with "no-follow" is hidden under up to 10 ads where relevancy is determined by who pays the most.

What is interesting is that the different methodologies are trying to get to relevancy for the searcher. Wonder which one works better in the eyes of the beholders.

Dave

Hey, I never noticed the "Link collections from experts and..." included in the Teoma SERPs... Especially if you are doing research. I checked them out for even some commercial terms and the link collections were almost all INFO stuff. That is top notch IMO - even if they don't have my site listed there. lol (But I do enjoy #1 in the Real Results.)

I might use Teoma next time I am researching.

Here is my pinch of salt

Local rank is used to reduce network and web server load, its used for search trees, its used for indexes, its used for loads of things. Its not a term specific to a particular search engines method.

When a search is triggered, the engine creates a list of results in order of relevance. It then reviews the local link structure to assess patterns in document relationships. Hubs are detected this way, or subcategories if you like. Amongst these, sites which have lots of links from others in their hub and also have inbound links from other hubs can be considered authoritative.

Its all very sensible really, Teoma works like this as do other systems, but its not ground breaking in any way, and a commonly use technique in topic detection and datamining just to get an idea of the how data is distributed. Very very simply put its like if I gave you 10 cards and asked you to pick your favorite and then your second favorite and so on. You'd base it on what you like (what you're looking for in essence).

Teoma ranks by the number of linking pages on the same subject as your search. This is easily manipulated which already makes it a very bad idea. Their index isn't fresh by using this method as new Web pages will not appear in the results, as it takes time for the creators of other Web pages to link to new resources.

“…The link structure of the Web is significantly more dynamic than the contents on the Web. Every week, about 25% new links are created. After a year, about 80% of the links on the Web are replaced with new ones. This result indicates that search engines need to update link-based ranking metrics very often…”

“.. From our experimental data, we could observe that the top 20% of the pages with the highest number of incoming links obtained 70% of the new links after 7 months, while the bottom 60% of the pages obtained virtually no new incoming links during that period…”

[ Cho et al., 04 ]

Teoma isn't fast enough to cope with this IMHO.

Randfish how do you manage to scrub up so good every single day? Who's your butler, who's your tailor?

As always an good topic to write about from my own perspective, although make sure you know that I like your post very much! Another mention to you I think!

Xan,

Thanks for your pinch. I knew I was missing an ingredient ;)

I think even Teoma would agree with you that their indexing and crawling is not as fast as they'd like it. But, they are the ones who pioneered this technology, so I'll give them some credit. It's no surprise to learn that all the search engines are using subject-specific popularity to some degree now, I just wonder who's doing it "for real" and who's more or less "faking it" by relying primarily on anchor text

The quote from Cho reminds me of Mike Grehan's "filthy linking rich". Those at the top get richer, while those at the bottom languish in anonymity. This tells me that now is the time to get to the top - it's getting harder every day.

This is exactly why you must BUY the top site - and be willing to pay beyond current value. Just pray that the rankings hold after you take over.

Rand:

I'm curious about the anchor text reference. By that are you curious about which SE relies on anchor text to approximate subject-specific popularity or are you referring to web sites using anchor text to climb in allinanchor and serps at google?

Dave

Dave,

Yes, I meant that I am interested to know which search engines use true measures of semantic connectivity and strong ontologies to discover communities and which ones use the computationally less-expensive method of relying primarily of anchor text to indicate what community a site/page fits into.

BTW - Using Teoma's "resources" to find links is, in my opinion, at the top of the list for link building tasks - they make it so easy to find the hubs, it should be part of link building 101 by now.

I realize that it is simply a coincidence, but Ask started spidering very heavily this morning...under a new agent name.

Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)

Hey there Rand,

I went to the annual BCS conference yesterday (british computing society). The subject was web and artificial intelligence. Ontologies, taxonomies, etc,...were all discussed as were semantically related documents, seeing as the hot topic in data has been semantic systems for some time. We weren't able to find any evidence of semantic work in Google's results, and many explained that Google used "methods inspired from artificial intelligence". For semantically related docs, you would need to be looking for cook book, and you would also find in a high position sites simply offering recipies, without the word cook book or cookbook anywhere on it at all.

Ontologies are the "backbone" in some cases, but the idea is to move away frm these as much as possible.

I will publish a low-down of the conference (which was also sponsored by thr royal signals), and I guess we could discuss what was said there too. This was for delegates only, so people were asked to attend, coming from BT, QinetiQ, MoD, T-Motion, Accenture, Logica,... It was well worth the trip and the dinner was good

Xan:

Could you expand on your first paragraph? Are you saying that for a search for "cook book" google's highest ranking sites would include sites with recipes but no mention of the keyword search. Does that mean that the recipe sites (or pages or both) are analyzed as being semantically close to cook book? Does that further mean that if a site is well optimized from google's perspective and has phrases or is topically focused on a subject sematically connected to the search phrase that it can show for that search prhase?

Dave

Put it this way:

If i want to find stuff to do with cookbooks, I can stick that in a semantic search engine.

What will pop out will be sites all related to my query, so recipir book, is just as acceptable as cookbook right? Perhaps a page is deemed much more interesting to me because it has a lot to do with food, ingredients, recipies and so on rather than just because it happens to have the word "cookbook" on it or "cook" "book".

If the entire site follows this gist - wehey! Its a good site. If just one or two pages of it (density) do, then perhaps there's better things out there.

Same for anything. Would you walk into a shop called "Julie's summer wear" or "Bob's shoe emporium" to buy sandles?

think about it that way.

Xan:

I'm ultimately interested in how google works for my sites.

For the cook book example, how is google bringing the recipe sites and/or pages high without the use of the word cook book?

Is this an example of some kind of semantic analysis and if so how does it work?

Dave