Google BERT Update. How the Natural Language Algorithm Affects You

Just last week Google made waves with its announcement of quantum supremacy – the claim that they have developed a quantum computer that has been demonstrated to solve, in a matter of days, a problem that a “classic” supercomputer would most likely take thousands of years to solve. Elsewhere, stories about robots taking our jobs abound, while machine learning seems to be on everyone’s mind.

But change is not always flashy or even visible. Last week, Google introduced a new update named BERT, which is characterized as a massive and the biggest step forward for search in the past 5 years, as well as one of the biggest steps forward in the history of search altogether. Yet, look and ask around in the SEO community and you’ll see very little that announces as much. Also, it is not yet very clear what Google’s BERT is targeting and how will the SEO landscape be influenced by this big update. So, let’s figure it all out!

1. What Is Google BERT Update?
2. What Is Google BERT Targeting?
3. Does Google BERT Affect SEO?
4. Does Google BERT Affect Content Marketing?

Do Androids Dream of Electric Sheep? It’s the title of a novel by visionary sci-fi writer Philip K. Dick. If it doesn’t sound familiar, you might better recognize it under its movie adaptation title, Blade Runner. Both book and movie concern themselves with questions on what it means to be human in an ever more technological world and how to (still) distinguish between humans and androids. 2014 indie sci-fi Ex-Machina, by director Alex Garland, asks a similar question by referencing the concept of the Turing test: if a robot were to pass as human to every other human in the universe, would it still be a robot? These are fascinating questions and luckily we can still ponder about them in sci-fi literature and film. We’re not there yet in real life, although one has to wonder how long will it take until the more trivial “I Am Not a Robot” captcha will get checked by a robot (it has).

Even though we are still far away from sheep-dreaming androids, we’re seeing constant progress in the way of more human-like computer interactions.

What Is the Google BERT Update?

Putting it simply, Google BERT is supposed to help a machine understand what the words in a sentence mean, but with all the nuances of context.

Yet, to respond to the question what does BERT mean? we need to talk in a more explanatory note.

BERT, which is what the latest and the biggest Google algorithm update is called, stands for Bidirectional Encoder Representations from Transformers, and is a deep learning algorithm related to natural language processing.

So, is BERT a language model? (geeky alert ahead)

Yes, we can say that it is a language model. Yet, you need to know that even if BERT is a new concept, is not hot new. The BERT concept was made public in 2018, in a paper published by researchers at Google Artificial Intelligence Language. 

According to Google researchers, “unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide
range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications.”

Language model pre-training has been shown to be effective for improving many natural language processing tasks. These include sentence-level tasks such as natural language inference and paraphrasing, which aim to predict the relationships between sentences by analyzing them holistically, as well as token-level tasks such as named entity recognition and question answering, where models are required to produce fine-grained output at the token level.

If we were to ask Google what the BERT name means, we’ll get to see a range of interesting results. In all fairness, my search query was for “bert name”, nothing related to the update. Yet, if we’re looking at the “people also ask section”, we get three different pieces of information. Not contradictory, not opposite, but different which might translate in confusion for the user. Will the BERT algorithm update solve this matter? Let’s pursue our investigation to find out.

Fun Fact: The Google BERT Update was launched on October 25, the same day Kanye West launched its latest album, Jesus Is King. 

What Is Google BERT Targeting?

By Google’s own estimates, BERT update will affect 10% of all queries. That’s a tremendous percentage, but it might not have caused a visible splash by SEO community standards. That’s most likely because the update focuses on “longer, more conversational queries”, whereas these longer tail queries are queries that (probably) SEOs don’t target as much in a heavy way.

If that last part sounds familiar, it might be because it’s not too far off from our recent discussions about search intent. The basic question is, then, what does the user really want to find out? There are quite a few examples out there illustrating the difference that BERT made.

Search Engine Journal provides an example of BERT understanding, using the phrase “how to catch a cow fishing,” which has nothing to do with the image that may be conjured in your head right now (or in the picture below) and everything to do with a very particular sense of the word “cow” in relation to fishing, referring to a large striped bass.

Google itself offers some examples of queries which would have been pretty clear in intent to a human conversation partner (e.g.: “2019 brazil traveler to usa needed a visa,” and “do estheticians stand a lot at work”), but were previous to the update entirely lost on Google, based on the results it displayed.

By applying the BERT models to both rankings and featured snippets in Search, Google pretends to be able to do a much better job, helping users find useful information. In fact, when it comes to ranking results, BERT will help Search better understand one in 10 searches in the U.S. in English, and we’ll bring this to more languages and locales over time.

Particularly for longer, more conversational queries, or searches where prepositions like “for” and “to” matter a lot to the meaning, Search will be able to understand the context of the words in the query. And users can search in a way that feels natural for them.

In an article from 2018, Rani Horev predicted BERT’s importance. As he stated, BERT will improve search and is undoubtedly a breakthrough in the use of Machine Learning for Natural Language Processing. The fact that it’s approachable and allows fast fine-tuning will likely allow a wide range of practical applications in the future. Follow KDnuggets for more information and resources on AI, Data Science, and Machine Learning. 

Does Google BERT Affect SEO?

Yes, the BERT update affects SEO and allow me to explain why. 

SEO – Search engine optimization is the process of making your site better for search engines. Therefore, any update that the search engines are making to their algorithm influences the search engine optimization process. 

Now, the question that remains is what can you do to optimize for the BERT update?

If we listen to Danny Sullivan, Google’s public Search Liaison, who helps people better understand search and helps Google better hear public feedback, the answer is pretty straightforward: nothing new. What Danny actually highlights is that there is nothing that you should do from today on that you shouldn’t have done before BERT. And that is: write content for users.  

We hope that there is no doubt for anyone that Google has been focusing on content for a couple of years (here’s a case study on Panda 4 update, which targeted content big time and affected lots of important websites). And we don’t want to re-iterate the “content is king” nor to over-highlight the importance of writing both SEO and user-friendly content. Yet, let’s try to understand where SEO is standing now in the context of BERT.

We believe that two main aspects need to be taken into consideration when we ask ourselves how the latest Google Update influence SEO. 

Identify and Optimize for the Right User’ Search Intent

In the BERT training process, the model receives pairs of sentences as input and learns to predict if the second sentence in the pair is the subsequent sentence in the original document. So, the algorithm is trying to better understand the user’s needs, even to predict them if and when possible. 

Google BERT update tries to (even) better understand the users’ search intent. 

Search intent or keyword intent is the reason why people conduct a specific search. Why are they searching? What are they trying to achieve through their search? Are they trying to figure out the answer to a question or do they want to reach a specific website? 

With the increasing use of mobile and voice search, where people need fast and contextual answers to their questions,  Google tries to become more and more able to determine the search intent of people. So, the whole Google SERP is now trying to best fit the search intent and not the exact searched keyword. Now, more than ever, there will be situations when the exact searched term will not even be included in the Google search results page. And this happens because Google has become better and better at determining the search intent of people.

Google has to figure out what exactly do people want, so it can offer them the search engine page results they need. And, from an SEO point of view, your job is to create content that is relevant to the Google users and matches their search intent. 

Not to linger on this anymore, remember that search intent is more important than ever.

Optimize for Featured Snippets

Google stated that BERT is about users’ natural language and understanding longer queries. What Google tries to highlight with the focus on featured snippets is that searcher intent is to find content that responds exactly to the this question really quick. 

Featured Snippets (also known as answer boxes, knowledge graphs or Google direct answers).  If you’re searching for something like “how many calories does an apple have”, you’ll get a direct answer, highlighted within a box, just like in the examples below. We did a really cool research on answer boxes a while ago; you should check it out.

Also, with the BERT update, Google focused on showing even more relevant featured snippets. To better understand what this improvement is all about, we are given the example the featured snippet for the query parking on a hill with no curb. Before, Google used to place too much importance on the word “curb” and ignored the word “no”, not understanding how critical that word was to appropriately respond to this query. So they would return results for parking on a hill with a curb. This latest update seems to fix this matter, as the search engine better understands the query and the context. 

But how should you optimize for the featured snippets or answer boxes? Well, here is where content steps in, so keep on reading. 

Does Google BERT Affect Content Marketing?

Yes, Google stated that content is even more important, and therefore, one should focus their full attention on writing content relevant for the user.

By definition, content marketing implies creating valuable, relevant, and consistent content to attract and retain a clearly defined audience. And Google’s featured snippet seems to endorse that. And since Google announced that it had leveraged its pre-trained language model BERT to dramatically improve the understanding of search queries, it’s clear that content marketing needs to comply with this biggest leap forward in the history of search. 

As Google better understands natural language, focusing on longer tail keywords and on featured snippets, it’s clear that there are big opportunities for content writers to serve their readers with content written more “humanly”, that answers a searcher’s question as quick as possible and provides much value. 

Leaving the theory aside, here are the steps you should take to write content that is relevant for your users but also content that will rank high so that your users will find it. 

Step 1. Perform a Google SERP analysis

Every keyword research or content optimization process should start with a SERP analysis together with a competitor analysis. The Ranking Analysis from the Content Optimizer Tool gives you tons of insights related to the analyzed keyword. Quick and easy, you get to know the search volumes, what type of content ranks on that keyword, how difficult it is to rank on that query (by following keyword difficulty), as well as how popular that keyword is among searchers. Also, the tool lets you know the exact keywords and links that boosted that page in the top Google results to easily optimize your content. 

Step 2. Create relevant & optimized content 

Content Optimizer does most of the job for you. I know I’m biased and I don’t want to praise the tool too much, but the reality is that it does most of the job for you. Once you performed the ranking analysis, what you need to do is start writing a new piece of content or optimize the existing one and the Content Assistant will let you know the exact keywords you need to use so your content will be relevant for the user’s search intent. 

Remember, you need to write for humans. BERT seems to make Google understand even better the searcher’s queries, so you have no excuses. 
And if you’re asking why do you need a tool to “write for humans” I’d tell you that a tool can give you lots of insights of what you’re users are actually interested in and you can write content that will answer their needs; and secondly, your users need to find your content first on the first Google page result to access it. 

Step 3. Discover new keywords & rankings opportunities

You don’t have to limit yourself to one targeted keyword;  you need to discover other queries that your users might be interested in. Searchers have more than one question when it comes to products from your business. Take the opportunity and offer them relevant content for most of their questions. You can use the same Content Optimizer for this task. The tool has two sections that will automatically let you know what other questions are related to your search query:

The Keyword Explorer  – this section is great for keyword analysis and for discovering new keyword opportunities. It also gives you the possibility of seeing only the question suggestions. Get inspired by the list of questions, check out the relevancy of the question, its volume, CPC (cost per click), and choose the one that is the most suitable and profitable.  

The People Also Ask section – the Content Assistant will let you know the exact keywords you should use in your content, what people are searching for, but it will also offer you a set of questions that relate to your original search query. You should consider answering these questions in your content or create new content starting from these questions. 

Although BERT integration in Google Search is currently only available for English queries in the US, Google says it is planning to apply BERT to additional languages and locations. So the rest of the world won’t have to wait too long until BERT will be responsible for the searches in dozens of languages. 

When applied to ranking and featured snippets in search, BERT models can process words in relation to all other words in a sentence rather than considering them one-by-one and in order. This enables a better “understanding” of context, which is particularly helpful when it comes to longer, more conversational queries, or searches where prepositions strongly affect meaning. This brings huge opportunities to the search world and big challenges to SEOs and digital marketers. 

And while the performance improvements are impressive, Google acknowledges that natural language understanding remains an ongoing challenge. This doesn’t mean that one shouldn’t adapt their marketing strategy or should re-think their marketing automation and SEO strategies to comply to today’s search marketing requirements. Yet, with search engines becoming more and more complex, there are no “complete guides” or “tips and tricks” lists to optimize for BERT or any other future updates (most likely). You need to keep yourself updated and have the user in mind no matter what you do.