Said on stage 22
Among the many signals Google calculates during indexing, the ones singled out as having large effects on search results were country, language, freshness, SafeSearch and spam.
Speaker GoogleEvidence transcript
- Extends D1-C037 Day 1: For classic Search, crawling means Googlebot, scheduling and robots.txt, with AI used in parts such as…
Country and language are among Google's most important signals and have been used since Google's early days.
“Both of them are extremely, extremely important”
Speaker GoogleEvidence transcript
- Extended by D3-C079 Day 3: To order candidates at retrieval, Google uses signals collected during indexing, and the first two are…
- Extended by D3-C082 Day 3: Country is the second retrieval signal: a user searching from Switzerland wants cheese from Switzerland, not…
Smaller languages have fewer documents on the web and so worse coverage, from an indexing perspective, than large ones, for example Basque compared with Spanish or Portuguese, Hungarian compared with German, and Sicilian compared with Italian.
Speaker GoogleEvidence transcript
Google detects the language of each document and weights language in index selection so that the index is not dominated by one or two languages.
Speaker GoogleEvidence transcript
- Extends D2-C585 Day 2: Google determines a page's language for indexing from the page content, not from a language code in the URL.
Without language or country input to index selection, Google's index would become an English index, because so many documents on the web are in English.
“if you just let the index decide what to rank without language or country input, then you would have an English index”
Speaker GoogleEvidence transcript
Google also takes country into account in index selection, so that countries producing less content in a shared language, such as the UK for English or Switzerland for German, are not disadvantaged against the US or Germany, which produce most of it.
Speaker GoogleEvidence transcript
In ranking, country and language signals help Google serve users the right content for their country and language.
Speaker GoogleEvidence transcript
- Extended by D3-C006 Day 3: Google's first step in understanding almost any query is to detect its language, which tells Google roughly…
- Extended by D3-C080 Day 3: At retrieval, Google tries to match results to the user's language wherever possible: someone searching in…
Freshness is a signal for queries that deserve fresh results ('query deserves freshness'): when a breaking event hits a city, such as possible closure of Barcelona's airport, users want really fresh results, not results from two weeks ago.
Speaker GoogleEvidence transcript
- Extends D1-C045 Day 1: Ranking signals differ by result type: web pages (text, links, passages), images (resolution, colour…
- Extended by D3-C487 Day 3: Google's ranking systems guide describes 'query deserves freshness' systems that show fresher content where…
Google calculates some freshness signals during indexing but uses them in ranking.
Speaker GoogleEvidence transcript
Freshness plays no big role in index selection: whether a page was published 30 years ago or today makes no difference to its chance of being indexed.
“It doesn't play a big role in index selection.”
Speaker GoogleEvidence transcript
Google uses multiple systems to protect users from unexpected explicit results: its algorithms detect whether a user is looking for explicit content and rank results accordingly, so the aim is not to never show explicit results but to avoid showing them to users who are not looking for them.
Speaker GoogleEvidence transcript
Whether a page's content is explicit does not matter for indexing: judged on content type alone, an explicit page has the same chance of being indexed as a major news site's homepage.
Speaker GoogleEvidence transcript
Google calculates SafeSearch signals during indexing rather than in ranking, because ranking happens online with no time for such calculations, while indexing has the processing power.
“In ranking, everything has to happen online, and there's just not enough time to calculate those things.”
Speaker GoogleEvidence transcript
Calculating SafeSearch signals for a page needs its whole parsed HTML plus knowledge of which pages link to it and which pages it links to.
Speaker GoogleEvidence transcript
Spam stops a document at indexing: when Google detects that a document is spam, it does not let it into the index.
“If we detect that something is spam, then we're not letting it into the index.”
Speaker GoogleEvidence transcript
Google uses more and more AI to detect spam, and Google's testing shows that this AI-based detection is highly accurate.
Speaker GoogleEvidence transcript
- Extends D1-C041 Day 1: Statistical models have been used at Google for over 20 years, for catching spam and originally for the 'Did…
- Extended by D3-C205 Day 3: Google's quality talk said AI has fundamentally changed how Google builds spam updates, letting it evaluate…
- Extended by D3-C206 Day 3: Google's quality talk said AI lets Google catch new types of spam and catch more of it.
SpamBrain, Google's AI-based spam detection system, is nowadays built on Gemini and fine-tuned specifically for finding spam.
“built on top of, well, nowadays, Gemini, and fine-tuned to that specific purpose of finding spam”
Speaker GoogleEvidence transcript
Used byglossary term SpamBrain
- Extends D1-C041 Day 1: Statistical models have been used at Google for over 20 years, for catching spam and originally for the 'Did…
- Extends D1-C039 Day 1: Gemini is not part of Search, but it uses crawlers for data, shares some technologies such as tokenization…
SpamBrain is central to Google's spam-fighting efforts and has been improved many times since its launch.
Speaker GoogleEvidence transcript
- Extended by D3-C287 Day 3: Google's spam updates page says its automated spam detection systems run constantly, and a notable…
SpamBrain was said to detect 5 times more spam sites than the spam algorithms Google had launched before it.
Speaker GoogleEvidence transcript
Google's testing shows that, thanks to SpamBrain, more than 99% of visits from Search are now spam-free.
Speaker GoogleEvidence transcript
- Repeated by D3-C275 Day 3: Google aims to keep more than 99% of search results free from spam and said it already achieves this, thanks…
Many of Google's signals are described in its developer documentation, though not always under the name 'signals'.
Speaker GoogleEvidence transcript
Combing Google's documentation for signals is not the best use of an SEO's time; creating content that users will like is a better one.
“If you are just focusing on actually creating the content that users will like, then that's probably a better use of your life.”
Speaker GoogleEvidence transcript
- Repeats D1-C059 Day 1: The opening keynote closed with the advice to think about UEO, user engine optimisation, next to SEO and GEO…
What Google's documentation says 5
Google's guidelines for sites with explicit content recommend grouping explicit pages on a separate domain or subdomain; otherwise Google's systems may judge the whole site explicit and filter all of it when SafeSearch is on.
Publisher Google Search Central
Used byrequirement DEV-IDX-12
Google's spam policies say that sites violating them may rank lower in results or not appear in results at all.
Publisher Google Search Central
Used byrequirement DEV-HTM-06
Google's 2021 webspam report says SpamBrain, its AI-based spam-prevention system, was launched in 2018 and has been continuously improved since.
Publisher Search Central blog (21 April 2022)
Used byglossary term SpamBrain
Google's 2022 webspam report says SpamBrain detected 5 times more spam sites in 2022 than in 2021, and 200 times more than when it first launched.
Publisher Search Central blog (11 April 2023)
Google's guidelines for sites with explicit content say to mark each explicit page with rating markup set to adult, either as a meta tag named rating or as an HTTP response header.
Publisher Google Search Central
Used byrequirement DEV-IDX-12
Analysis by the author 6
A site serving a smaller country in a big language, such as British English or Swiss German, should make that country unmistakable (country-code domain or region-specific hreflang, local address, prices and currency) so its pages can benefit from index selection's country balancing instead of competing with the much larger US or German content pool.
Author Ibrahim Anjro
Changing a page's date or making cosmetic edits will not help an old page get indexed, because freshness plays no big role in index selection; freshness pays off only when ranking for queries that deserve fresh results.
Author Ibrahim Anjro
Because explicit content on its own does not lower a page's chance of being indexed, the practical risk for a site that mixes explicit and general content is SafeSearch classification, which looks at the whole page and its links; keep explicit pages on a separate domain or subdomain, as Google advises.
Author Ibrahim Anjro
Used byrequirement DEV-IDX-12
The SpamBrain launch year mentioned on stage, with hesitation, was 2022, which does not match Google's documented 2018; 2022 is the year of the improvements described in Google's 2022 webspam report, so cite 2018 as the launch year.
Author Ibrahim Anjro
The '5 times more spam sites' figure said on stage matches Google's 2022 webspam report, but the report compares 2022 with 2021 (and gives 200 times since launch), not SpamBrain with earlier algorithms; quote the documented comparison.
Author Ibrahim Anjro
Language, country, SafeSearch and spam signals, and some freshness signals, are calculated when a page is indexed, so a fix such as correcting a page's language or removing content that triggers SafeSearch only counts once Google recrawls and reprocesses the page; request recrawling of the most important URLs after the fix.
Author Ibrahim Anjro
Used byrequirements DEV-IDX-12, DEV-MON-02