Fundamentals of The Semantic Search

This post is a brief introduction to the semantic search and technologies associated with it.

So, what is semantic search?

Chances are you have already witnessed the magic of semantic search. Every time you search for an ambiguous term and think to yourself that your next step would be to refine the search, Google somehow manages to pull the best result for you. It’s almost as if it knows what you are looking for.

Say hello to Semantic Search, a technology that is re-defining the way we find our answers on the web and an important step towards smarter Search Engines. By smart I mean, smarter than humans.

Let’s grab the official definition of Semantic Search (as recognized by Google).

Semantic search seeks to improve search accuracy by understanding searcher intent and the contextual meaning of terms as they appear in the searchable dataspace, whether on the Web or within a closed system, to generate more relevant results.

Definition credit: Wikipedia

The definition above raises questions like how the searcher intent and contextual meaning are determined. The closed system is a reference to the closed loop control system, a system that takes current output into consideration and makes corrections if necessary based on feedback. The feedback bit is made possible by the use of ajax. Something that you have very likely already witnessed while typing a query into Google’s search field.

The short clip above shows Google’s closed loop control system in action. As you can see, I was suggested the most popular or the most relevant (personalized) searches while I was typing. When I changed the letters, it quickly corrected the suggested items based on the feedback.

Google’s semantic indexing sees Melbourne as an object. If you have had some exposure to object orientated programming, think of the items suggested in this example – weather, airport, map and CBD as it’s properties. In Semantics, such a mesh of object – property relations is known as the semantic network. This network collects and interrelates the information from diverse reputable sources.

In 2012, Google named this network Knowledge Graph.

Knowledge Graph: The foundation of Semantic Search

Knowledge Graph is essentially a collection of information about information. A Search Engine’s semantic network identifies objects in the data that have been indexed and gathers information around that data. The result is a many to many mapping between the objects and their ‘facts’.

People, TV shows, movies, cities, countries or sporting events, all are seen as objects, as we would in the real world and associate knowledge to these objects.

I recently watched Marvel’s Daredevil. Let’s look it up and see what facts have been associated to it.

TV show knowledge graph - semantic search

The sidebar shows a whole lot of information about the show. In this instance, the majority of information seems to have been fetched from IMDB, a reputed source.

Google’s semantic network has grown hugely and so has the amount of sources it considers reputed. I think it is safe to say that there’s a lot more to come and this is just the beginning.

The facts about the objects are often presented to the users in the form of cards, also known as answer boxes. Sometimes you can also see the related facts in ‘people also ask’ accordion.

For example, if I look for Australia’s national game, I get this:Google's answer box card

and then this, when I scroll down the page a bit:

related questions

A user interested in finding out about the sport culture in Australia will find this information extremely useful as it will help him broaden the scope of what to look for.

 

Other Applications of Semantic Network

It is because of the knowledge Graph that Google Now, Google voice search and Google’s predictive search are so effective. The closed loop control system mentioned earlier in this post takes your feedback into consideration and BAM!, modifies it on the go making the entire user-machine interaction interactive.

This kind of control system employs fuzzy logic. Fuzzy systems are capable of not dealing in absolutes, just like humans. In other words, there doesn’t have to be a true or false answer to the complicated questions that we deal with everyday. Instead, we can have an answer calculated as the best (personalized) answer for us.

To demonstrate why we need fuzzy logic to mimic human behavior, let’s go to the field of robotics and query the AI powered operating system of a robot:

Is this glass full?

Unlike the conventional logic, where the answer will be either true or false, a fuzzy logic driven robot will assess the amount of water in the glass allocate it a membership grade and let us know that the glass is 85% full.

The set of data is arranged in fuzzy sets (as opposed to crisp sets). You can see the difference between the two of them in the figure below:

membership function of a fuzzy set Credit: Wikipedia

membership function of a fuzzy set Image Credit: Wikipedia

Fuzzy sets and anything else fuzzy mentioned thus far allows systems to make decisions like humans. This comes in handy in the field of natural language processing as that used by the google translate app on all major smartphone platforms. An app like that acts as a translator while another human speaks a language that you do not understand. The more we use it, the better it gets as Google gets the feedback and is able to make adjustments to it’s decision tables, a set of conditions and actions that impact the output.

This Google Translate app demo shows the recent upgrade that the Google Translate app received. The demo shows how it also comes with the excellent image processing capabilities.

Such on-the-go translations would be very hard to implement with a traditional approach not to mention how useless they would be while processing a language spoken by humans, something that doesn’t have a rigid syntax like the programming languages. Knowledge graph and fuzzy logic, in conjugation, have offered a great solution to this problem.

Understanding user intent and context

Semantic search is particularly instrumental in understanding the user intent and context of a query. It is what makes your experience on Google magical. If you are a travel freak and often look up travel destinations, you will be shown hotels and flights related information when you look up ‘Hanoi’, however if you are interested in games or have been playing tower of Hanoi, you may get a suggestion for ‘tower of Hanoi’ when you look it up.

Another example, if you have looked up and watched the movie Rio, you may automatically get suggestions that are related to TV Entertainment, however if you live in Australia and have worked for the mining giant Rio Tinto, you may see that as your first result, travel enthusiasts may see Rio de Jeneiro pushed up for them.

Semantic indexing allows Google to understand an object’s context of use. Data on a user’s search history or location paired with other users’ behavior in his or her area, allows it to know the intent.

This knowledge becomes a critical factor that Google and other semantic search engines use very well to determine the position of a web page in the SERPs based on a user’s intent.

Understanding your audience’s true intent is the key to a good SEO and content marketing strategy, something I’ll be discussing in the future posts.

Gist

This article shows how semantic web combined with fuzzy decision making is a good solution to improve the search accuracy, understanding the searcher intent and providing the contextual meaning of the queries. The semantic search engines are driven by big data. Google churns billions of search queries into meaningful predictive search or corrections during the process of translation, everyday. The data from search behavior is utilized to improve the fuzzy decision tables.

With the quantum computing era knocking at our doors, the day semantic web powered Google outsmarts us is not far away. The autonomous operating systems of spaceships and robots that we have been witnessing int he sci-fi movies will soon be a part of everyday household.

Fast forward a few decades and we will be using smart bionic extensions, capable of tapping into our brain (I know, the possibilities are endless) and extending our neural capabilities.

Semantic web is a step towards everything I have mentioned in the last two paragraphs as it will be providing the all important knowledge.

Enjoyed this post? Let me know your thoughts on semantic web in the comment section below.

Speak Your Mind

*