Knowledge base v2.13.0 · Community edition · data through 2 October 2026
Thing · Directive
A meta tag in a page's HTML that gives search engines page-level rules such as noindex, nofollow or nosnippet.
Glossary · Robots meta tag
An HTML tag in the head of a page that gives search engines page-specific rules, such as noindex or nosnippet, for indexing and display in results.
Documented in
Google's robots meta tag specification tells site owners to place the tag in the head section but notes that Google Search does not enforce that placement and also respects robots meta tags in the body of an HTML document.
Google Search Central · Day 2 · Controlling indexing
Google's robots meta tag specification says robots meta tags and X-Robots-Tag headers are found only when a URL is crawled, so the rules on a URL disallowed in robots.txt are never seen and are ignored.
Google Search Central · Day 2 · Controlling indexing
Google's robots meta tag specification says that with nosnippet a static image thumbnail, if available, may still be shown when that results in a better user experience.
Google Search Central · Day 2 · Controlling indexing
Google's robots meta tag specification says data-nosnippet may be extracted both before and after rendering, so the attribute should not be added to or removed from existing elements with JavaScript.
Google Search Central · Day 2 · Controlling indexing
Google's robots meta tag specification says Google chooses the snippet length when no max-snippet rule is set, and with max-snippet:-1 chooses the length it believes most effective; it does not say that -1 produces longer snippets.
Google Search Central · Day 2 · Controlling indexing
Google's robots meta tag specification says Googlebot considerably decreases the crawl rate of a URL after the date and time set in its unavailable_after rule.
Google Search Central · Day 2 · Controlling indexing
Google's JavaScript SEO basics guide says that when Google encounters a noindex rule it may skip rendering and JavaScript execution, so using JavaScript to change or remove a noindex robots meta tag may not work as expected.
Google Search Central · Day 2 · Controlling indexing
Google's JavaScript SEO guide says Googlebot sends every page with a 200 HTTP status code to the rendering queue, whether or not it contains JavaScript, unless a robots meta tag or header tells Google not to index it, and Google uses the rendered HTML to index the page.
Google Search Central · Day 2 · Lightning session D: Rendering and JavaScript
Google's JavaScript SEO basics says every page with a 200 status code is queued for rendering, whether or not it uses JavaScript, unless a robots meta tag or header says not to index it; for non-200 pages such as 404 error pages, rendering might be skipped.
Google Search Central · Day 2 · What is Google friendly JavaScript
For client-side rendered single-page apps, where meaningful status codes can be impossible or impractical, Google's documentation gives two ways to avoid soft 404s: a JavaScript redirect to a URL that returns a 404 status, or a noindex robots meta tag added with JavaScript.
Google Search Central · Day 2 · What is Google friendly JavaScript
Google's robots meta tag specification says the nosnippet rule also prevents a page's content from being used as a direct input for AI Overviews and AI Mode, and max-snippet also limits how much of it may be used that way.
Google Search Central · Day 2 · How does the index look like?
Slide and stage claims that name it, the ones Google’s documentation does not cover first.
Google estimated that meta annotations, such as robots meta tags, are processed in typically 45 to 90 minutes, with a minimum of 200 seconds (about three minutes) and an end point of 1 to 4 days.
John Mueller said there is currently no RFC that standardises robots meta tags, unlike robots.txt.
John Mueller · Day 2 · Controlling indexing
John Mueller opened with a true-or-false quiz slide asking whether robots meta tags can make a page more visible in Search results than having none, and later answered that the statement is true.
John Mueller · Day 2 · Controlling indexing
Setting the max-image-preview robots meta tag to large can make content perform surprisingly well in Discover, Gary Illyes said.
Gary Illyes · Day 2 · Using images to your advantage and Engaging Search users with videos
The robots meta tag is a piece of HTML code in the head section of a page that gives search engine crawlers page-specific instructions.
John Mueller · Day 2 · Controlling indexing
The primary function of the robots meta tag is to control how a page is indexed and shown in search results.
John Mueller · Day 2 · Controlling indexing
The robots meta tag is written as <meta name="robots" content="rule1,rule2">, or with the name googlebot instead of robots, with several rules separated by commas.
John Mueller · Day 2 · Controlling indexing
John Mueller said robots.txt does not control indexing, so robots meta tags are what site owners have to use to control it.
John Mueller · Day 2 · Controlling indexing
The robots meta rule all is the default value and does nothing: it places no restrictions on indexing the page or following its links, the same as having no robots meta tag, and it is not an instruction that search engines must index the page.
John Mueller · Day 2 · Controlling indexing
John Mueller recommended putting robots meta tags directly in a page's HTML exactly as intended, and adding them with JavaScript only where that is not possible, as in a JavaScript web app.
John Mueller · Day 2 · Controlling indexing
The noimageindex robots meta tag tells Google not to index the images on the page.
Gary Illyes · Day 2 · Using images to your advantage and Engaging Search users with videos
John Mueller encouraged attendees to help write a specification for robots meta tags, saying internet standards groups are made up of ordinary people and need only passion and a willingness to join the discussions.
John Mueller · Day 2 · Controlling indexing
John Mueller suspected that if robots meta tags were reinvented today, the page-level nofollow rule would probably not be part of them.
John Mueller · Day 2 · Controlling indexing
Audit robots meta tags set by templates and plug-ins: an explicit all rule does nothing and can go, while a page-level nofollow strips link signals from every link on the page and is better replaced by qualifying only the specific links that need it.
Ibrahim Anjro · Day 2 · Controlling indexing
Put everything indexing depends on (title, meta description, canonical, robots meta tag, main text and links) in the raw HTML so it is available without rendering, let JavaScript add enhancements only, and check the rendered HTML in URL Inspection for the rest.
Ibrahim Anjro · Day 2 · Lightning session D: Rendering and JavaScript
Do not set noimageindex on video watch pages: Gary Illyes said it also stops the video, through its thumbnail, and although the robots meta tag specification mentions only images, the Video indexing report treats a missing or blocked thumbnail as a reason a video is not indexed.
Ibrahim Anjro · Day 2 · Using images to your advantage and Engaging Search users with videos
Kit items about Robots meta tag: their own words name it, or several of the claims they rest on do.
Do not add, change or remove robots meta tags with JavaScript
Rests on 9 claims, 4 of them naming Robots meta tag; its own words name Robots meta tag
Allow full snippets and large previews with max-snippet:-1 and max-image-preview:large
Rests on 8 claims, 3 of them naming Robots meta tag; its own words name Robots meta tag
Prefer rel attributes on individual links to a page-level nofollow
Rests on 6 claims, 2 of them naming Robots meta tag
Server-render the main content and everything indexing depends on (SSR, static generation or hybrid)
Rests on 11 claims, 2 of them naming Robots meta tag
Return a real 404 for unknown routes in single-page apps, or use a documented client-side fallback
Rests on 7 claims, 1 of them naming Robots meta tag; its own words name Robots meta tag
Also inglossary terms Robots meta tag, max-snippet, noimageindex, X-Robots-Tag
Things named in the same claim, with the number of claims they share.