Before proceeding further, it is helpful to examine the concepts of synonymy and polysemy. Yes, these are more odd-looking words, but they have presented challenges to the science of LSA and LSI.

Synonymy:

Synonyms are different words that describe the same idea. For example, the words auto, roadster, vehicle, dragster, hot rod, “wheels,” and automobile alldescribe an internal combustion travel method we call a car. Synonymy is the
practice of using multiple synonyms to describe a single idea.

Under current (non-LSI) search engine rules, a search for “car” might actually fail to display relevant web pages that deal with auto, roadster, vehicle, dragster, hot rod, wheels, or automobile if those exact words do not appear on the page! Under LSI, all those synonyms form an interconnected index of words that leave no
doubt to the search engines that the web page is about cars. That's why LSI is important.

Let's repeat that. Once the search engines begin relying heavily on LSA and LSI to determine the relevance of your web pages, and your web page about cars contains the word “car” twenty times, people searching for auto, vehicle, hot rod, dragster or roadster probably won't find your page.

On the other hand, if you rewrite your page in the LSI style to include the synonyms auto, roadster, vehicle, hot rod, “wheels,” automobile, dragster and other synonyms for “car,” the page should rank well for the word “car,” and may even rank well for some of those synonyms.

“Hot rod, wheels, dragster and roadster” are all car synonyms that imply fast and powerful cars. Depending on the content of your page, the search engine's LSI program might also determine that your page is related to “car racing” too!

Imagine how powerful it would be to be ranked simply on a concept that is extrapolated from the words on your page!

Polysemy:

Polly who? Not who, but what.

Polysemy is the situation in which one word has multiple meanings. A non LSI search engine will probably return multiple but unrelated pages in which the word is used in the wrong context.

A classic example of this concerns snakes, specifically the python. If you are researching the python, your search results will undoubtedly contain multiple listings for the British comedy troupe Monty Python. The “Monty Python” pages
are relevant for the word “python,” but not for the concept of “snake.”

If the same search engine used LSI, it would know that python refers to a long constrictor snake. It would also know that Monty Python refers to comedy. The search results would display information about snakes and omit the Monty Python references.

Yes, that capability exists now by excluding the word Monty from the search query, such as: “python -Monty.” However, you are doing the LSI in your head when you do that, because the non-LSI search engines are currently too
uneducated to know that Monty Python is not a snake!

Another example is the word “tree.” There are many types of people who would search for the word “tree,” such as botanists, genealogists, and computer scientists. However, each of those people would expect very different search
results! The computer scientist doesn't want web pages about oaks and pines, just as the botanist's time will be wasted on pages related to family trees.

Humans can very quickly determine what the topic is if only synonyms are used (synonymy). Likewise, humans can easily determine which word meaning is the correct one by the context (polysemy). Teaching computers to do this as easily is quite a challenge.


Part 3 of LSA and LSI will be posted ina a couple of days.