Latent Semantic Indexing!

There seems to be some consensus on the fact that Google is starting to give much more importance than ever to Latent Semantic Indexing since the last update.

Should we start to modify our sites and mold our new projects to this effect?

No. Its unlikely LSI is used.

See SEW threads and search science for more info.

Hi,
Google is following LSI (Latent Semantic Indexing).
What is the benefit of Latent Semantic Indexing ?
Give your helpful suggestiong.

Thanks

Google is NOT using Latent Sematic Indexing right now. Search engines may have trouble with the context of a search but its highly unlikely that this is in place to rectify the issue. So don't worry about it unless you want to learn more about it. It is quite interesting and understandable after some study. I had the oppourtunity to explain to a 15 year kid on the plane to New York a couple weeks ago. He was interested in the term matrix that can be created using LSI. He loved it.

Care to share any of that info with us?

http://javelina.cet.middlebury.edu/lsa/out/cover_page.htm

I would love to, but got to jet out the door. So here is one article that is very good regarding LSI. Its long, so its not a quick one, plus if you are not familiar with some of the concepts they mention, it might take a couple reads.

LSI is in use on several sites that I know about, but these are mainly portals. Extending beyond these to a major search engines has its obstacles.

Also check out the stuff we have on SEroundtable.com

http://www.seroundtable.com/archives/001481.html

Any article which conforms that google ain't using LSI? What makes you so sure.. I'm currently doing my masters program in LSA/LSI as a search tool.. Tell me if it sounds hopeless and perhaps good reasons or refer to articles...

Songz...

How sure are you about google not using LSI? Am currently doing research on LSA for search engine optimization... Please refer articles about this.. Is there a hope for me?

Well i would love to chip in on this conversation but i havent got a a clue what you are on about ;)

LSI is really important when you have several keywords that are related by category by not by association regarding that acutal word structure. It seems to me that this could be important for directory listings regarding geographic content.
I think one of the main reasons that it will become important is that it can give you an outcome that is not just related to the keyword searched, but also the entire set of keywords associated with the search term. For instance, if you searched for "Jessica Simpson" you may also find things about nick lachey, louis vuitton, mtv, etc...
Do you guys see google using this algorithim in the future? - I could see it easily being implemented into the GMail system. I am pretty sure that Teoma already does the LSI based approach.

-Scott Fish

I believe Xan is probably most qualified to answer this, but if she's busy, you should try posting in the topic at SEW - LSI/A is more thoroughly discussed there. I'm slightly too busy to do a complete repsonse right now sorry!

Hi all,

ah...the old LSI issue has resurfaced again I see!!

First off, LSI is a statistical measure used to address the problems of synonomy, and polysemy. The idea is to retrieve documents based on the concepts in the document.

All well and nice. So why is no one raving about it if its the solution to end all our problems??

because:

Single value decomposition technique is used, which is unstable and and impractical for large highly dynamic collections (the web). The reason is that it needs to be run every time something changes.

There is no consensus as to how many concept dimensions to use. Researchers like Deerwester and Dumais just proceeded by trial and error, which is not possible with a huge corpus of web data.

After a certain threshold, the performance steadily decreases.

With LSI, you can't really use complicated keywords, because these will be filtered out the SVD stage. They cannot lead to generalization so they get discarded.

The SVD method has been used by excite for a long time. However not LSI.

Additionally, the patent is owned by Bellcore:

Telecordia Technologies (Bellcore) Patent : Computer information retrieval using latent semantic structure (U. S. Patent No. 4,839,853, June 13, 1989) before initiating any commerical product development based on LSI.

hehe...the patent cafe use it even: iamcafe.com/

Saying that Google uses LSI is like saying that a bike has a bolt on it.

You're talking about some of the brightest minds in computing. Do you think that they would use a technique shown to have real drawbacks for use on the web? I have read some solid research out of some of these institutions when I've had to implement some hard techniques, and I can tell you it would be lovely if it read like LSI! In a digital library environment it has worked ok, well enough in fact. So the very core idea is retained and a different method is made, which in turn, is just another cog in the machine.

A lot of SEO people believe it is used, which is fine. It can't hurt you or your site. Just don't treat it as the holy grail and the most important thing that Google does.

For me LSI is a rough and ready plug if I have to quickly get some estimates on a corpus.

Not a pretty site but has working examples for you to try: lsa.colorado.edu/

BTW, LSA and LSI are exactly the same thing.

Happy working