Day 2: Indexing 41
Shown on screen 6
A crawler report shown on screen drew one canonical group of five URLs as a link graph with the leader marked by a crown, and listed each member's HTTP status, indexability, document language, leader flag, incoming and outgoing links, and hints.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence slide photo, transcript
The crawler report shown on screen ran group-level checks on each canonical group: an overall group status, whether the leader has self links, whether languages are consistent across members, and whether the group contains a chain.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence slide photo
In the real-world canonical group shown on screen (a German-language travel magazine section, site not named), a feed URL's canonical pointed to a URL on another subdomain that 301-redirected to the leader, a canonical chain through a server-side redirect.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence slide photo
The crawler report shown on screen rated a canonical group member that issues a redirect instead of displaying content as an error, and flagged canonical links whose target returns a non-200 status or is itself a redirect.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence slide photo
The crawler report shown on screen flagged a URL discovered only through a canonical tag, with no internal a href links leading to it, as a phantom document outside the visible site structure.
“a phantom document not part of the visible site structure”
Wording checked against the slide or recording
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence slide photo
The crawler report shown on screen flagged a canonical chain as a problem: canonical links that form a chain instead of all pointing directly to the group's canonical leader.
“Canonical links form a chain rather than all pointing directly to the leader.”
Wording checked against the slide or recording
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence slide photo
Said on stage 22
Google's speaker suggested checking rel=canonical links with a crawler such as Screaming Frog to make sure they are reasonable.
Speaker John MuellerIn Day 2, 11:55 · Handling web duplicationEvidence transcript
Used byrequirement DEV-MON-06
- Extended by D2-C408 Day 2: A community speaker advised auditing canonicals as whole canonical groups rather than pair by pair, paying…
Broken canonical tags can make the wrong pages of a site show up in search results.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-MON-06
- Extends D1-C113 Day 1: The real cost of duplicates is control and crawling: Google may choose a canonical you did not want, and…
According to the canonical link specification cited by a community speaker, when a canonical tag is declared improperly, the application that processes it may apply its own heuristic and decide what to do instead.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
- Repeats D2-C380 Day 2: Many SEOs assume Google must follow their rel=canonical, but because people sometimes get it wrong, Google…
- Extended by D2-C873 Day 2: A community speaker said that, under the canonical link specification, an improperly declared canonical tag…
Checking canonicals pairwise (page A has a canonical to page B, page B a self-referencing canonical) is common but, in a community speaker's experience, misses the bigger picture, such as other links pointing to A or B.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
In a community speaker's terms, a canonical group is the set of all URLs connected through canonical links, and its canonical leader is the page that all the group's canonical links ultimately point to.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
A community speaker advised auditing canonicals as whole canonical groups rather than pair by pair, paying special attention to each group's canonical leader.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-MON-06
- Extends D1-C128 Day 1: Google's canonicalization guide says some duplicate content on a site is normal and not a violation of its…
- Extends D2-C398 Day 2: Google's speaker suggested checking rel=canonical links with a crawler such as Screaming Frog to make sure…
A community speaker said it helps to view small canonical groups as a graph that shows any redirects between members, together with the indexability of each member page.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Multiple canonical declarations on one page conflict and are invalid, so the search engine applies its own heuristic and picks the canonical for the site owner; a page should declare only one canonical.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-03
Canonical chains, where a page's canonical target leads on to yet another URL, are named as improper use in the canonical link specification, according to a community speaker.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-04glossary term Canonical chain and canonical loop
Canonical chains can run through canonical links alone, through a server-side redirect between members, or through a client-side redirect between members.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Canonical loops, such as two HTML pages whose canonical tags point to each other, are a structural conflict: the group has no canonical leader, so its canonical information cannot be used.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-04glossary term Canonical chain and canonical loop
Canonical loops can also run through server-side or client-side redirects, not only through canonical tags.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
A community speaker said that at best a search engine's heuristic would treat a canonical loop through redirects as self-referencing canonicals, and doubted that this is often done when the loop runs through a client-side redirect.
“But regarding the client-side redirect, I highly doubt that this is often done.”
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
When internal links point only to page A and the canonical leader is reached only through A's canonical link, the leader is reachable by machines but not by human visitors, a signal conflict that asks the search engine to index a page users cannot reach.
“you're telling the search engine to index something that a human can't reach”
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-04
- Extends D1-C067 Day 1: A page with no internal links depends on sitemaps alone to be found, so it is discovered slowly and attracts…
For a canonical leader that users cannot reach through links, a community speaker suggested revisiting the canonical graph and probably making the internally linked page the leader instead.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
A community speaker said pages carry different link equity, and a canonical leader that is not the strongest page in its group is technically valid but most likely suboptimal for ranking, so the strongest page should be the leader.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
- Contradicts D2-C348 Day 2: Google forwards the signals attached to every URL in a duplicate cluster, such as links, to the…
The canonical leader of a group should be indexable: it should carry no noindex robots directive, return no error status code and not be blocked in robots.txt.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-04
Canonical links between different language versions of a page are a misuse: if a search engine accepted such a canonical group, only one language version would rank.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-06
Language versions of a page should be connected with hreflang annotations instead of canonical links.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-06
In a community speaker's experience, real-world canonical graphs are far more complex than the isolated examples in the talk and commonly mix several canonical problems at once.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
A community speaker advised fixing canonical chains by cleaning up the canonical graph so that every canonical link points directly to the canonical leader.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-04
A community speaker said that, under the canonical link specification, an improperly declared canonical tag can also be ignored completely by the application that processes it, not only replaced by that application's own heuristic.
Speaker Tobias SchwarzIn Day 2, 12:05 · Lightning session E: Managing Duplicates and Site MovesEvidence transcript
Used byrequirement DEV-CAN-03
- Extends D2-C402 Day 2: According to the canonical link specification cited by a community speaker, when a canonical tag is declared…
What Google's documentation says 6
Google's guide to specifying canonical URLs says Google supports explicit rel=canonical link annotations as described in RFC 6596, the canonical link relation specification.
Publisher Google Search CentralAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Google's 2009 Search Central blog post introducing rel=canonical says a canonical URL may be a URL that redirects: Google then processes the redirect as usual.
Publisher Search Central blog (12 February 2009)Annotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Google's 2013 Search Central blog post on rel=canonical mistakes says to specify no more than one rel=canonical per page: when a page has more than one, Google will likely ignore all of them, and any benefit of a legitimate canonical is lost.
Publisher Search Central blog (8 April 2013)Annotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-CAN-03
Google's 2009 Search Central blog post introducing rel=canonical says Google's algorithm is lenient and can follow canonical chains, but strongly recommends updating links to point to a single canonical page for optimal canonicalization.
Publisher Search Central blog (12 February 2009)Annotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-CAN-04glossary term Canonical chain and canonical loop
Google's guide to specifying canonical URLs says a canonical helps consolidate signals for duplicate pages: links to a duplicate URL are consolidated with links to the preferred URL once the preferred URL becomes canonical.
Publisher Google Search CentralAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-CAN-05
Google's guide to specifying canonical URLs says that on pages using hreflang, the canonical should be a page in the same language, or the best possible substitute language if no canonical page exists in the same language.
Publisher Google Search CentralAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-CAN-06
Analysis by the author 7
A broken canonical setup does not fail visibly: pages still load normally while the choice of URL passes to the search engine's own heuristics, so canonical tags need a regular audit with a crawler and the URL Inspection tool.
Author Ibrahim AnjroAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-MON-06
To audit canonicals at scale, resolve every crawled URL to the final URL its canonical links and redirects lead to, group URLs by that leader, and flag groups with more than one hop, a loop, several canonicals on one page, or a leader that is noindexed, blocked, non-200 or redirecting.
Author Ibrahim AnjroAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirements DEV-CAN-04, DEV-MON-06
After a migration or a subdomain change, update rel=canonical tags to the new final URLs: canonicals still pointing at old, redirecting URLs create chains through server-side redirects like the one in the real-world example shown.
Author Ibrahim AnjroAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-CAN-04
A leader reachable only through a canonical and a leader with weak link equity share one fix: point internal links, sitemap entries and redirects at the URL chosen as canonical, so the leader is both reachable for users and the strongest page in its group.
Author Ibrahim AnjroAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-CAN-05
Check that every canonical group contains a single document language: a group spanning languages means canonicals are joining versions that hreflang should connect, and each language version should keep its own self-referencing canonical.
Author Ibrahim AnjroAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Used byrequirement DEV-CAN-06
The crawler report shown at the talk rated a redirecting canonical target as an error, while Google's 2009 guidance accepts a canonical URL that redirects and says Google can follow canonical chains; the gap is one of severity, since both recommend pointing every canonical link straight at the final URL.
Author Ibrahim AnjroAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves
Google says canonicalization consolidates link signals from duplicates into the chosen canonical, so a leader with few links of its own is not necessarily weaker once Google accepts it; the practical risk of a weakly linked leader is that Google, treating rel=canonical as a hint, picks a different page as canonical.
Author Ibrahim AnjroAnnotates Day 2, 12:05 · Lightning session E: Managing Duplicates and Site Moves