Knowledge base v2.13.0 · Community edition · data through 2 October 2026
Thing · Directive
An HTTP response header that carries robots rules such as noindex, so they also work for PDFs, images and other non-HTML files.
Glossary · X-Robots-Tag
An HTTP header that carries the same rules as the robots meta tag, also for files that are not HTML. Like the tag, Google only sees it on URLs it is allowed to crawl.
Documented in
Google's robots meta tag specification says robots meta tags and X-Robots-Tag headers are found only when a URL is crawled, so the rules on a URL disallowed in robots.txt are never seen and are ignored.
Google Search Central · Day 2 · Controlling indexing
Google's Search documentation gives notranslate as a rule for a robots or googlebot meta tag or an X-Robots-Tag header, and says the rule opts a page out of all translation features in Google Search.
Google Search Central · Day 2 · Controlling indexing
Google's documented ways to keep a site's images out of search results are a robots.txt disallow rule (for example for Googlebot-Image) or a noindex X-Robots-Tag HTTP header, with the Removals tool for emergencies.
Google Search Central · Day 2 · Using images to your advantage and Engaging Search users with videos
Hiding an image as a CSS background is a fragile way to keep it out of Google: it only stops extraction from that page, so the same image URL used in an img element elsewhere or listed in a sitemap can still be indexed; the documented robots.txt or noindex X-Robots-Tag methods are the reliable route.
Ibrahim Anjro · Day 2 · Using images to your advantage and Engaging Search users with videos
Kit items about X-Robots-Tag: their own words name it, or several of the claims they rest on do.
Send robots rules for PDFs, images and other non-HTML files in the X-Robots-Tag header
Rests on 3 claims, 3 of them naming X-Robots-Tag; its own words name X-Robots-Tag
Keep images out of Search with robots.txt or X-Robots-Tag, not with CSS tricks
Rests on 5 claims, 2 of them naming X-Robots-Tag; its own words name X-Robots-Tag
Rests on 8 claims, 1 of them naming X-Robots-Tag; its own words name X-Robots-Tag
Also inglossary term X-Robots-Tag
Things named in the same claim, with the number of claims they share.