Page 2 - Advanced On/Off-Page Optimization for Engines using Semantic Analysis

Glad to see we're getting into this. I really am.

The tools behind this are not too far off. My developer says we'll have them done by the 28th. However, one part that is totally easy is the C-Index calculcation. It's literally Z divided by X+Y-Z... I know you can all add and subtract and most of us can divide, so...

In Excel, let's say you have three columns. The first(A1) has the number of search results for the keyword term or phrase you're targeting (remember to search in quotes for multi-word terms). The second(B1) has the number of search results for the term/phrase you want to check (to see if they're semantically related). The third column(C1) has the number of search results for both terms/phrases together - "kw 1" "kw 2" type of search.

Then in column D, you calculate the C-Index parts per thousand. You can literally copy and paste this into Excel: =(C1/(A1+B1-C1))*1000

Column D now has your answer. The higher it is, the closer the two words/terms are related in Google's index. Ta da!

If anyone still has questions about this process or why it's important, don't hesitate to ask. As the pack used to say "We'll be here all week"

Rand:

I'm working on this now. For my site, there was a surprising change in google for several of my money terms. What was amazing was that some sites that (in my mind) have high relevancy...but had miserable rankings shot to the top...along with my site (although in a couple of cases) my site is now 2nd for these terms where before it was always first...and the relevant competitors were way behind. These money terms are relatively competition free. Frankly I'm interested in getting back to #1 across the board and planting a couple of my interior pages between my site and those of competitors.

Of additional interest, the "sleeping bags" test page at digital point forum plummeted. As I'm sure you know, that site's ranking was purely based on massive anchor text bls. Not only did it crash in serps but its allinanchor ranking crashed.

I've read a bit through the LSA/LSI thread at SEW. Interesting and very provocative.

Please explain more how this formula and analyses can help with rankings as you see it.

Its all very interesting. As you may know my site represents a local business. It ranks high for what has been my main focus till recently; yadayada school. It ranked across the board #1 for my money terms which are all local/regional; Maryland yadayada school, DC yadayada school, etc and also for every variation on that. Now that isn't the universal case.

With this latest change these competitors floated to the top similar to my site. Frankly these several sites are the only relevant sites on the web for the topic. Everything else is realistically peripheral. Prior to the change, for some regional phrases the top page would include a page or 2 from my site and a number of pages that linked to my site.

IMHO, simply based on a few observations googles new methodology has excellent relevance and it could well be based to a large extent on relevancy in language, both for the site in question and for links to the site in question. If so the issue is how do we get on top of that.

Dave

Dave,

I'm happy to talk more on the subject of the reasons why you should use these. Thanks for bringing it up.

First off, let's define LSA/I once and for all - it is a single, particular methodology proposed by IR (that's Information Retrieval - a precursor to the search engine industry, and still the name used by the academic world) specialists several years ago. LSA does NOT refer to the subjects we're talking about here - which is the use of related keywords in general. So, let's try to be careful about the way we use LSA/I, as I'd hate to get a bunch of readers confused.

Now, on to the the Purpose:

We surmise, as SEOs, that Google especially and Yahoo! & MSN probably are using more on-page analysis and more textual/semantic analysis of the documents in their index (web pages). They do this because it sucks big red monkey bottom when you have a bunch of spam/irrelevant pages come up at the top of a search query because spammers have used global site popularity (i.e. PageRank) and anchor text to boost a page to the top of the results.

So, how do you combat this?

Well, if you're a search engine, you make sure that not only is a page/site on-topic, you make sure that the pages/sites that link to it are as well. We've referred to this before as greater 'theming', but it's much more than just by category (like the Google site-flavored search). Search engines have way of calculating and extracting the most relevant/on-topic terms in a document. But, they're really just more advanced versions of the C-Index that I illustrated at the top of this post. So, we can use our simplistic method and come away with at least a basic idea of what to do.

What the C-Index tells you is HOW RELATED TWO PHRASES/WORDS REALLY ARE. They do this by comparing the co-occurence of the terms/phrases (that means the number of times they happen on one page together). Hence, the C-Index (Co-Occurence Index). So, we, as SEOs can use this knowledge and the equation to get a rough idea of not only whether OUR pages are on-topic, but we can also discover if the page we're about to request/cajole/buy a link from is on the topic too! Pretty sweet, huh?

Let's give a nice, big for example:

AvatarFinancial.com - my site. It's on the topic of Non-Conforming Commercial Real Estate Loans. Basically, if you have a commercial property that needs to be financed, and the bank say NO, you can submit a loan with Avatar. They're great guys, very honest people in a very slimey industry. I love working with them. (OK, enough self-promotion).

Avatar is what people call a 'hard money lender' or a 'bridge loan lender'. The two big terms to keep in mind are "hard money" and "bridge loans". We want Avatar's pages and links to be 'on-topic' for these subjects.

So, I conduct some searches, and pull up 100-200 terms and phrases to check against these two phrases. I put them into my Excel spreadsheet, with the nice little equation I showed above, and voila! I've got a list of the highest C-Index phrases for the subject. Now it's on to do a little 'on-topic' analysis, which for me is just logically asking myself - is this really 'on-topic' for me, or are these just co-occuring by happenstance?

Then, I look at the list of terms I'm left with and their C-Index numbers:

hard money | bridge loans 78.04 | 66.01
hard money | lenders 12.53 | 11.20
hard money | commercial real estate19.29 | 10.22
hard money | commercial loans 71.54 | 29.15
hard money | direct lender 35.05 | 9.98
hard money | commercial mortgage 63.59 | 36.30
hard money | non conforming 20.28 | 13.75
hard money | business loan 56.27 | 22.14
bridge loan | non conforming 15.43 | 17.09
bridge loan | mortgage lender 23.97 | 16.46
bridge loan | commercial mortgage 52.44 | 38.17
bridge loan | mortgage banker 31.12 48.79
bridge loan | mortgage broker 18.96 | 18.48
bridge loan | hard money 51.00 | 30.88
bridge loans | mortgage lender 21.91 | 9.57
bridge loans | commercial mortgage 50.83 | 23.15
bridge loans | direct lender 80.58 | 27.95
bridge loans | hard money 68.51 | 65.15

Above, we've got my excel chart, showing the highest c-index for 'on-topic' terms with each of my 2 phrases (first number shows Google, second uses Yahoo! - just to be sure). There's a lot of good terms/phrases there with C-Index 10 and 20+. So, as I go out to find links and submit my site, I don't want to spam or stuff, but I do want to make sure that I use these terms along with my target terms when getting anchor text, links, making pages, etc.

Hope that helps! As always, ask if you need help or an explanation.

A good copyrighter can do wonders with the right data points in mind. It's one thing to say "write me an article on hard money" and another thing to say "write me an article focusing on hard money, direct lender, bridge loans..." -- and of course, this does make tailored link exchange campaigns a lot more effective. Some of us have been preaching to vary the anchor text for a while...you can devise a very thorough plan based on those correlations (especially once you tie it in to the right keywords to target, competitive strengths, etc).

I like it Randfish...another good concept to pull out of the bag o' tricks, whether looking at a website's expansion or the development of a new site.

Cygnus

I am developing the beta versions of the tools today. I hope they can help people to use this information more easily, I know it's a pain in the butt to do it manually.

Hi Rand

As ussual an excellent post. I remember reading something similar a couple of months or years ago on Danny Sullivan's forum. He had some Prof or something, that was discussing the same sciencxe there. It just had allot more spaceships in it lol confusing me after a couple of days reading it. Your explanation is fantastic. In the casino industry we are competing against each other on all levels of the playing field. The old Blackhat, whitehat story. These strategies are certainly worth looking at and will give the sites with actual content a much better chance. I think I might considder crossing borders again

amazing....im going to plug this into a spreadsheet and do some evals...thanks

#1 LSA/LSI is very unlikely to be used by a search engine seeing its too expensive, and too slow. It would be impossible to sustain in a dynamic environment.

#2 Semantics is ultra, ultra complex, and computational semantics are not the same as formal semantics.

I will write something about it on the blog, because I feel there's a lot of underestimation and confusion over what it is and what techniques are possible other than the very obvious ones, which are used, but there's more efficient methods in this stochastic environmet.

You're basically right Randfish as far as on-page content is concerned, that is the way to look after it properly and the method works. That's not how SE's determine it though. I'll try and explain.

Thanks Xan..

I may end up quoting your blog in this space if that's alright with you.
I'm glad there's agreement on LSA as well. I just recently read a paper describing the process (although the equation sections were over my head) and felt like the principles of LSA are something that all SEs must use, but the specific technique is something different and shouldn't be confused with semantic connectivity, which is what we are really talking about.

Once again, though, thrilled to have you here Xan.

No worries randfish, I'm glad the blog is of use. I know it gets a ton of traffic and glad its not on my servers!

Theres one blog or something where i am quoted and there is a picture of an old crusty man with 60's plastic glasses on...thats not me, I swear! I thought it was very funny though!!

As I said if I can help with something in particular I will write something about it, at the moment I check what the flavour of the week is and clean up things if need be.

Its been interesting being involved in the SEO community so far, and I was suprised I admit to be welcomed so warmly everywhere. I would however have much less of a warm welcome if anyone around here found out. Odd that your own would be more hostile....oh dear, getting philosophical again...

Currently I'm of the opinion that the topic of this thread tends to describe the recent changes in google, at least as far as I see them. I'm too new to these ideas and have no experience with IR or semantic analysis or any of these concepts speak of them knowledgeably, but the sites that I have been following seem to have risen or dropped for certain keywords based on some combination of on-page optimization and connectivity through links from relevant pages. It also seems as if a high level of these factors has been able to supplant sites with dramatically more backlinks, including those with specific anchor text targetted for keyword phrases.

Any further comments, agreement, disagreement or explanations are greatly appreciated.

Dave

Dave,

This thread's advice probably is more useful at Google than the other engines - they appear to be using advanced semantics more than other engines at this point. However, I wouldn't be surprised to find this very applicable to both MSN & Yahoo! in the next 6-9 months. As far as the latest GG update... it's still anybody's guess, but why would you ignore the possibility that the optimization practices above could help you rank better???

FYI i've written semantics now.