Last year, potpiegirl found a document from Google, by searching the Google index, that was entitled “2011 Google Quality Raters Handbook”. Today doing the same thing, I found a new leaked, 161 pages Google document entitled :
“General Guidelines Version 3.27 – June 22, 2012”
I will not provide a link to this document, due to obvious reasons, but you can try to search, either on Google or the Social Media sites and you will probably find it.
What I will share with you in this post, is my opinion on the contents of this document.
The document is focused on
- identifying webspam (technical analysis – keyword stuffing,redirects etc/ authority of the blog,site/ commercial intent – thin affiliates)
- webpage rating (content type/ language/ flags)
- geo-location webpage rating (if a page is geographically related to the query)
- query (user intent rating)
Aproximately 60 pages talk about the “Rating Guidelines” and explain to the person that is going to be the rater the concepts. A few of them are:
- Understanding the Query
- Classification of User Intent: Action, Information, and Navigation – “Do-Know-Go”
- The Rating Scale
- Vital Pages for People Queries
The next 16 pages are about “URL Rating Tasks with User Locations”.
32+19 pages talk about “Page Quality Rating Guidelines” & “Rating Examples”.
Here is a screenshot of the Page Quality rating task page:
One particular thing that is really interesting, is an example about “Named Entity Queries”.
Snapshot talks by itself.
Part 5 (15 pages) of the document is about “Webspam Guidelines”.
This looks at things like:
- Technical Signals
- Fake vs Ok sites comparisons
- Commercial intent of webspam pages
The last chapter is about the EWQQ that evaluation system that the rater will use.
Part 7 and 8 are :
- Quick Guide to URL Rating
- Quick Guide to Webspam Recognition
To conclude this shows us that Google still uses an army of raters to train their algorithms and validate situations that the algorithm can not do by itself.
It is a pure user oriented view, from Google’s side and everything is related to the User Experience. This is used both for natural search and paid search but it seems they are mostly focused on the natural part.
Reading the document will provide both insight into the Google mindset and a technical view on some of the aspects of SEO.
Can we confirm in any way that this document is in fact from Google? I’m just looking through it now, and although the primary resources being cited are Google properties, I couldn’t find any overt statement of Google ownership. Thoughts?
if you performa the following searches in Google you will find some interesting documents :
“General Guidelines Version 3.27 – June 22, 2012″
Taking in consideration the latest version from 2011 which resembles quite well and the amount of data and things that are found in the document, it is hard to believe that this would be a hoax. It is a 161 document. Why would someone write such an indepth document with “white hat” stuff that relates to alost everything speculated and talked in the SEO industry. It would not make to much sense. I do not see whom it would favor such a document … unless this would be one of the agents’ 777 stuff.
It just seems really convenient. Has Google ever acknowledged any of these documents, including the previous one initially reported by potpiegirl? I know it doesn’t make much sense for this to belong to another entity – just keeping my skepticism intact, regardless of how excited I am for the invaluable information in that document.
yes it did by asking her to remove it. she published it on the blog and was contacted by them to have it removed.
Hi Razvan,
I was waiting for this for quite some time. Recently (about a month a ago) new Quality Rates Guidelines fell in my hands. These were newer the the one potpiegirl leaked (Dec. 2011 – V. 3.20), and it already featured the page quality sections as well as the named entity queries. I found it very strange that no blog wrote about them, as I thought more people might have received them.
Interestingly there are 12 (!) additional Google Quality Raters Documents (e.g. Local Rating Guidelines, Image Rating Guidelines, Mobile Rating Guidelines…). As far as I know these where not published anywhere yet – correct me if I am wrong.
I published the whole list with version numbers and dates here: http://buzzmatic.net/google-quality-raters-guidelines-leak-2012 (The article is in german but the list is in English obviously).
there are indeed other documents. I am not aware of all those 12 but I know that others exist in the wild. good stuff Raoul.
[...] More on that later. Below, a look at the new section of the rater’s handbook, which was written about yesterday by Razvan Gavrilas on [...]
I find it interesting that the rating itself is now pushed toward rating of BOTH website as a whole and the landing page for the query. There’s actually even a third type of a rating that they should give input on: the homepage rating.
How in the world are they going to grasp the purpose as well as the reputation (WTF?) of the website as a whole if they have only just landed on a single page? Does anyone with access to this document see any instructions for the raters to acquire that knowledge?
I don’t see how this can be done with any degree of scalability: it sounds like the rater should now visit at the very least three pages of a site before they can enter their input: query landing page -> homepage -> a random (?) page. The last step would presumably be needed to judge whether the query landing page was the only good page on the entire site. Wouldn’t you want to know that before you enter your input on the site as a whole? And what can you learn about a site from three pages, anyway?
Google’s lucky the raters are (most likely) not unionised: they have just been given a 200% workload increase. I hope they get paid accordingly.
Anyhow, jokes aside, I would be grateful if someone could shed some light on what are the raters’ guidelines for finding the purpose and the reputation the site as a whole.
Thanks!
I don’t understand why these guidelines aren’t made available to the public; they would help us all understand how to build better websites, instead of just guessing what needs to be done. And no, we wouldn’t “game” the system, since learning how to create websites that have high quality content and offer the visitors a good experience doesn’t have anything with manipulating the search engine results.
Thanks, it’s really useful to compare the updated version with the last one, especially after all the black and white animals evolving so rapidly.
It’s often difficult to cite a source for advice when a client asks for one; we recently worked on a web site where all the images were hosted on a separate domain, of course I advised the client that they’d do better to have their own original image content hosted on their domain, now I read on page 161 under “Thin Affiliates”: “Check the “properties” of images on the page…Check to see if the address of the image is the same as the address of the page..”
tons of details
. you can’t be paranoic enough when being an affiliate 
coming from searchengineland.com reading the current updates I find the complete documente to be a must read for all SEOs – no matter this has been deliberately leaked or not.
Does not contradict anything Google always preaches…
So this form is for manual reviews. I would have to say it would be virtually impossible to apply this manually to all websites, especially when you consider that it is doe for every page AND every keyword that page is trying/applicable to rank for.
I wonder how many of these guidelines Google are able to apply algorithmically?
welll. they don’t . but for highly competitive terms in the top 100 they surely are.
and also for flagged sites by their algorithm for whatever reason.
OK, thank you. I just googled the document ;o) and will read it tonight. I think mainly the same, but in parts hopefully something new or inspiring! Best regards. Ralf
Matt Cutts insists that these ratings don’t “directly” affect the search engine results. I find this hard to believe if these scores did not have an affect what would be the point in hiring a small army of raters, and paying them to rate every site on the internet.
they might not influence directly but they do indirectly. how I see this is like training an algorithm. we practically did the same thing with some of the algorithm we have on detecting the types of sites, links etc.
any AI algorithm needs to be trained and this means a large set of validates data that goes into its input, based on which that algorithm tries to make similar decisions.
I have been a Google rater during 6 months. Tired of being paid 7 euros the hour and UNDERPRESSURE because you have to rate more than 120 webs(weeks. I discover with this new Mnaula that currently Raters have to write comments about sites… Unbelievable ! I don’t know how they find time and be profitable. In fact, with european criters, it is very bad hourly rate but maybe for others countries , it can be a well paid work !!!
I confirm this is the manual that google raters receive to learn how to rate web sites.
since the comment is anonymous and we can not confirm your identity, I can not say much here … but only that this is a pure opinion.
when you say 120 sites per week how many hours do that means ? A quality review from an “expert” does not take probably more than 5-15 minutes per site to give those “basic” scores.
Hello Razvan, this is my first visit to your blog, glad to have dropped by! My understanding of this post is, provide quality content and you have nothing to worry about; Is that over simplifying it?
that is the basic idea that Google guides itself nowadays. Quality Content that generates organic backlinks.
I do think that the manual is being applied to see if the Google algoritm may be improved. It is impossible to review all the new pages and blogs that appear every day.
Good website, Razvan.
It is no news, that there was a leak of the new Guidelines for Webmasters, that was like four weeks ago … but soon after the wrongly and early posted new Guidelines, the Google-member deleted it. Even most blog entries with way more sreenshots did disappear.
Finally having new Guidelines, plus our own findings how to get out of a bad situation, when you were hit by the Penguin Update, is not to bad. I thinks it´s quite helpful. And even if it can´t be done, reviewing all new pages and blogs, Google shows some good will.
Plus, there are so many SEOs and Marketers complaining about Google, but still we seem to hang on to it. Why not use Bing more often, specially after the results of a blind-test show, that many users favour the Bing-SERPs.
Jennifer,
This was posted on September 6 2012. We were among the first to publish about this.
Bing is a great search engine .Unfortunately it does not the market share Google does . If it will Google will be left in the dark by the SEO community.