Google advises against using noindex to save crawl budget and against using robots.txt to temporarily reallocate budget. Use robots.txt only for pages you never want crawled, and 404 or 410 for removed pages.
Thing · Directive
noindex
A robots meta tag or X-Robots-Tag rule that tells Google not to show a page in search results.
- Claims
- 29
- In Google’s docs
- 7
- Said at the event
- 14
- Not in docs
- 3
- Kit items
- 11
Glossary · noindex
A robots rule that tells Google not to show a page in search results. It works only if Google can crawl the page and see the rule.
Google’s documentation 7
Documented in
Google's noindex documentation says the noindex rule works only if the page is not blocked by robots.txt: a crawler that cannot fetch the page never sees the rule, and the page can still appear in search results, for example if other pages link to it.
Google Search Central · Day 2 · Welcome to indexing day!
Google's JavaScript SEO basics guide says that when Google encounters a noindex rule it may skip rendering and JavaScript execution, so using JavaScript to change or remove a noindex robots meta tag may not work as expected.
Google Search Central · Day 2 · Controlling indexing
Google's JavaScript guides say that when client-side routing makes a real 404 status impractical, a single-page app can avoid soft 404s by redirecting with JavaScript to a URL whose server returns 404, or by adding a robots noindex meta tag with JavaScript.
Google Search Central · Day 2 · Lightning session D: Rendering and JavaScript
For client-side rendered single-page apps, where meaningful status codes can be impossible or impractical, Google's documentation gives two ways to avoid soft 404s: a JavaScript redirect to a URL that returns a 404 status, or a noindex robots meta tag added with JavaScript.
Google Search Central · Day 2 · What is Google friendly JavaScript
Google's documented ways to keep a site's images out of search results are a robots.txt disallow rule (for example for Googlebot-Image) or a noindex X-Robots-Tag HTTP header, with the Removals tool for emergencies.
Google Search Central · Day 2 · Using images to your advantage and Engaging Search users with videos
Google's Removals tool help says a temporary removal request usually takes up to a day to process and lasts only about six months, so the content must also be removed permanently, for example with a 404 or 410, a noindex or a password.
Google Search Console Help · Day 3 · How long does it take to..?
Said at the event 14
Slide and stage claims that name it, the ones Google’s documentation does not cover first.
Label
Grade
Day
Not in docs 3
Not in docsD2-C020
Google answered on a Q&A slide that it does not treat a robots.txt disallow as a noindex because some extremely important sites disallow their most important pages, by accident or out of ignorance.
Google illustrated why it does not treat a robots.txt disallow as noindex with an extremely important site, such as a national tax authority, that blocks very important PDF files with robots.txt: Google cannot index the PDFs' content but can at least show their URLs in search results.
Not in docsD2-C062
A slide named internal admin pages, temporary landing pages and thin content as typical use cases for the noindex rule.
John Mueller · Day 2 · Controlling indexing
Consistent with docs 6
Consistent with docsD2-C108
Removing a robots restriction such as noindex with JavaScript does not work, a slide said.
John Mueller · Day 2 · Controlling indexing
When Google finds a noindex rule in a page's HTML, it drops the page without even processing its JavaScript, so a script cannot switch the page back to indexable, John Mueller said.
John Mueller · Day 2 · Controlling indexing
The canonical leader of a group should be indexable: it should carry no noindex robots directive, return no error status code and not be blocked in robots.txt.
Tobias Schwarz · Day 2 · Lightning session E: Managing Duplicates and Site Moves
Index selection applies the negative signals that immediately block indexing: noindex (the likely reading of one unclear word), expired unavailable_after dates, soft 404s, non-canonical duplicates, spam signals and other policies.
Index selection drops a document carrying a noindex rule if it was not already dropped earlier in processing (noindex is a likely but not certain reading of the transcript, supported by the later mention of noindex among the Page indexing report reasons).
After Google processes a page that now returns a 404 or a noindex, it usually removes the page from its serving index within one to three weeks, sometimes much sooner.
Gary Illyes · Day 3 · How long does it take to..?
Confirmed by docs 4
Confirmed by docsD1-C106
The noindex rule consumes crawl budget, because Google must fetch the page to see it.
Confirmed by docsD2-C061
The noindex robots rule tells Google not to show the page in search results.
John Mueller · Day 2 · Controlling indexing
The robots rule none is equivalent to noindex plus nofollow, so a page that carries it will not show up in Search, provided Google can see the tag.
John Mueller · Day 2 · Controlling indexing
Not-indexed reasons in the Page indexing report include pages excluded by a noindex rule and 'Alternate page with proper canonical tag'.
Nothing to verify 1
An audience member noted that a robots.txt block does not mean noindex and that explaining this to developers is a constant struggle, and asked why Google does not treat a robots.txt block as a noindex.
From the audience · Day 2 · Welcome to indexing day!
Press and analysis 8
AnalysisD1-C110
A URL blocked in robots.txt can still be indexed without its content if other pages link to it, and Google cannot see a noindex on a page it is not allowed to fetch.
Ibrahim Anjro · Day 1 · How Google thinks about crawl budget
AnalysisD2-C023
Explain robots.txt and noindex to developers as two separate controls: robots.txt controls crawling, noindex controls indexing. To keep a page out of Search, let Google crawl it and serve noindex; a robots.txt disallow alone can leave the bare URL in results.
Ibrahim Anjro · Day 2 · Welcome to indexing day!
AnalysisD2-C849
Google's robots.txt introduction names links from elsewhere on the web as the reason a disallowed URL can still be indexed, while on stage Google spoke of the URL's importance; either way, well-linked important URLs are the disallowed ones most likely to appear in results, so keep such pages crawlable with noindex if they must stay out of Search.
Ibrahim Anjro · Day 2 · Welcome to indexing day!
AnalysisD2-C109
Ship robots rules in the HTML the server sends and never rely on JavaScript to lift a noindex: Google may skip rendering a page that arrives with noindex, so the page can stay out of the index even if a script removes the tag later.
Ibrahim Anjro · Day 2 · Controlling indexing
AnalysisD2-C855
On stage the rule was absolute (Google won't even process the JavaScript of a page served with noindex), while Google's guide says only that it may skip rendering; either way, a noindex in the served HTML must never be one that JavaScript is expected to lift.
Ibrahim Anjro · Day 2 · Controlling indexing
AnalysisD2-C193
In a single-page app, let the server return 404 for unknown routes where possible; otherwise use one of the two client-side fixes Google documents for error views: a JavaScript redirect to a URL that returns 404, or a robots noindex added with JavaScript.
Ibrahim Anjro · Day 2 · Lightning session D: Rendering and JavaScript
AnalysisD2-C530
Hiding an image as a CSS background is a fragile way to keep it out of Google: it only stops extraction from that page, so the same image URL used in an img element elsewhere or listed in a sitemap can still be indexed; the documented robots.txt or noindex X-Robots-Tag methods are the reliable route.
Ibrahim Anjro · Day 2 · Using images to your advantage and Engaging Search users with videos
AnalysisD3-C653
To hide a URL from Google Search urgently, use the Removals tool in Search Console (about 2 hours) and also add a noindex or a 404, which takes one to three weeks to drop the page from the index.
Ibrahim Anjro · Day 3 · How long does it take to..?
Built on these claims 11
Kit items about noindex: their own words name it, or several of the claims they rest on do.
Developer requirements 8
Use noindex to keep a page out of Search, and leave that URL crawlable
Rests on 12 claims, 6 of them naming noindex; its own words name noindex
Do not add, change or remove robots meta tags with JavaScript
Rests on 9 claims, 4 of them naming noindex; its own words name noindex
For an urgent removal, use the Removals tool in Search Console and also return 404 or noindex
Rests on 5 claims, 3 of them naming noindex; its own words name noindex
Return a real 404 for unknown routes in single-page apps, or use a documented client-side fallback
Rests on 7 claims, 2 of them naming noindex; its own words name noindex
Do not use noindex, nofollow or crawl-delay to save crawl budget
Rests on 6 claims, 2 of them naming noindex; its own words name noindex
3 more
Keep images out of Search with robots.txt or X-Robots-Tag, not with CSS tricks
Rests on 5 claims, 2 of them naming noindex; its own words name noindex
Point every canonical directly at a final URL that returns 200, is indexable and is linked
Rests on 8 claims, 1 of them naming noindex; its own words name noindex
Plan releases and fixes around Google's processing times, and judge results only after them
Rests on 11 claims, 1 of them naming noindex; its own words name noindex
Facts 1
- DocumentedF-025
Rests on 2 claims, 1 of them naming noindex; its own words name noindex
Also inglossary terms noindex, Robots meta tag
Connected things 21
Relations
- Part of Robots meta tag structure, no claim needed
- Affects Index selection
2 claims, 2 documented
Index selection applies the negative signals that immediately block indexing: noindex (the likely reading of one unclear word), expired unavailable_after dates, soft 404s, non-canonical duplicates, spam signals and other policies.
Index selection drops a document carrying a noindex rule if it was not already dropped earlier in processing (noindex is a likely but not certain reading of the transcript, supported by the later mention of noindex among the Page indexing report reasons).
Most often named with it
Things named in the same claim, with the number of claims they share.
- robots.txt 11
- JavaScript 8
- 404 7
- Disallow 4
- Redirects 3
- Rendering 3
- Single-page app 3
- Soft 404 3
- 410 2
- Crawl budget 2
- Index selection 2
- Page indexing report 2
- Robots meta tag 2
- X-Robots-Tag 2
- Canonicalization 1
- nofollow 1
3 more things
Topics that feature it
- Robots meta tags and noindex 15
- Directives and crawl budget 11
- JavaScript SEO pitfalls and fixes 8
- robots.txt rules 7
- How Google renders pages 3
- How long Google's processes take 3
- Index selection: deciding what gets indexed 3
- Soft 404s 3
- Images 2
- Canonical graphs: chains, loops and leaders 1
- Indexing statuses in Search Console 1