If faceted URLs must be crawled, use the standard & separator, keep filters in a consistent order, and return a 404 when a filter combination has no results.
Thing · Status code
404
The HTTP status code for a page that does not exist; Google drops such URLs from the index and crawls them less often.
- Claims
- 24
- In Google’s docs
- 7
- Said at the event
- 9
- Not in docs
- 0
- Kit items
- 10
Google’s documentation 7
Documented in
Google advises against using noindex to save crawl budget and against using robots.txt to temporarily reallocate budget. Use robots.txt only for pages you never want crawled, and 404 or 410 for removed pages.
Google's JavaScript guides say that when client-side routing makes a real 404 status impractical, a single-page app can avoid soft 404s by redirecting with JavaScript to a URL whose server returns 404, or by adding a robots noindex meta tag with JavaScript.
Google Search Central · Day 2 · Lightning session D: Rendering and JavaScript
Google's JavaScript SEO basics says every page with a 200 status code is queued for rendering, whether or not it uses JavaScript, unless a robots meta tag or header says not to index it; for non-200 pages such as 404 error pages, rendering might be skipped.
Google Search Central · Day 2 · What is Google friendly JavaScript
For client-side rendered single-page apps, where meaningful status codes can be impossible or impractical, Google's documentation gives two ways to avoid soft 404s: a JavaScript redirect to a URL that returns a 404 status, or a noindex robots meta tag added with JavaScript.
Google Search Central · Day 2 · What is Google friendly JavaScript
Google's site move guide says not to redirect many old URLs to one irrelevant destination such as the new site's home page, which can confuse users and might be treated as a soft 404, and to return a 404 or 410 for deleted or merged content that is not moved to the new site.
Google Search Central · Day 2 · Lightning session E: Managing Duplicates and Site Moves
Google's Removals tool help says a temporary removal request usually takes up to a day to process and lasts only about six months, so the content must also be removed permanently, for example with a 404 or 410, a noindex or a password.
Google Search Console Help · Day 3 · How long does it take to..?
Said at the event 9
Slide and stage claims that name it, the ones Google’s documentation does not cover first.
Label
Grade
Day
Consistent with docs 5
Google has recently seen more 403 responses to its crawlers (described on stage as 'authentication required'); it treats them as client errors, technically equivalent to 404 or 410, and drops those pages from Search and its AI features.
Gary Illyes · Day 1 · How crawling errors affect Search
Google treats a 402 Payment Required response as a 404 Not Found, because Googlebot cannot pay for content.
Gary Illyes · Day 1 · How crawling errors affect Search
The fix for a client-side soft 404 is to make a missing page end with a real HTTP 404 status code instead of 200.
Rebecca Yu · Day 2 · Lightning session D: Rendering and JavaScript
The fix for a client-side soft 404 is to serve a real 404 error page where appropriate; how to detect URLs that do not exist depends on how the app works and where it is hosted.
Erin Sparling · Day 2 · What is Google friendly JavaScript
After Google processes a page that now returns a 404 or a noindex, it usually removes the page from its serving index within one to three weeks, sometimes much sooner.
Gary Illyes · Day 3 · How long does it take to..?
Confirmed by docs 4
404 Not Found and 410 Gone both tell Google there is nothing at the URL, so the URL is not indexable.
Cherry Prommawin · Day 1 · How crawling errors affect Search
A soft 404 is a 404 in disguise: the page returns 200 but its content says something like 'page not found', information the site should have sent as the HTTP status.
Cherry Prommawin · Day 1 · How crawling errors affect Search
In single-page apps, a missing page often shows a custom 404 page while the server returns HTTP 200, because the front-end router, not the server, handles the 404.
Rebecca Yu · Day 2 · Lightning session D: Rendering and JavaScript
In an app routed with the History API, when a user reaches a page that does not exist, Erin Sparling suggested asking how they got there and redirecting to a real 404 page.
Erin Sparling · Day 2 · What is Google friendly JavaScript
Press and analysis 8
AnalysisD1-C372
Do not answer verified Googlebot with 401, 402 or 403, for example from a login wall, a paywall or a pay-per-crawl setup: Google treats them like 404 and drops the pages, and Google's status code page says not to use 401 or 403 to limit crawling; to slow crawling temporarily, return 429 or 503.
Ibrahim Anjro · Day 1 · How crawling errors affect Search
AnalysisD1-C419
A 404 fetch is still a fetch: Google's 2017 crawl budget post says generally any URL Googlebot crawls counts towards a site's crawl budget, so the stage point that 4xx responses do not affect crawl budget is best read as 'they do not slow crawling, and a 404 tells Google to crawl that URL less over time'.
Ibrahim Anjro · Day 1 · How Google thinks about crawl budget
AnalysisD2-C193
In a single-page app, let the server return 404 for unknown routes where possible; otherwise use one of the two client-side fixes Google documents for error views: a JavaScript redirect to a URL that returns 404, or a robots noindex added with JavaScript.
Ibrahim Anjro · Day 2 · Lightning session D: Rendering and JavaScript
AnalysisD2-C295
After moving to real paths, set the server or hosting rewrite rules so a direct request to every valid path returns 200 with the content (ideally server-rendered) and an unknown path returns a 404 status, which removes client-side soft 404s at the source.
Ibrahim Anjro · Day 2 · What is Google friendly JavaScript
AnalysisD2-C342
Return real error status codes for error states, 404 or 410 for missing content and 503 for outages such as a failed database connection, also in single-page apps; a 200 page whose main content is only an error message is treated as a soft 404 even when header and navigation look normal.
Ibrahim Anjro · Day 2 · Understanding what's on a page
AnalysisD2-C372
Make location and variant pages differ in their main content (local stock, staff, prices, addresses) and return 404 for empty or invalid combinations; otherwise every URL that fits the pattern can be folded into one canonical.
Ibrahim Anjro · Day 2 · Handling web duplication
AnalysisD2-C913
For URLs removed in a consolidation, return 404 or 410: Google's site move guide names those two codes, and Google's crawlers treat every 4xx code except 429 the same way, as content that does not exist, so the choice between them matters less than not redirecting to an unrelated page.
Ibrahim Anjro · Day 2 · Lightning session E: Managing Duplicates and Site Moves
AnalysisD3-C653
To hide a URL from Google Search urgently, use the Removals tool in Search Console (about 2 hours) and also add a noindex or a 404, which takes one to three weeks to drop the page from the index.
Ibrahim Anjro · Day 3 · How long does it take to..?
Built on these claims 10
Kit items about 404: their own words name it, or several of the claims they rest on do.
Developer requirements 10
Return a real 404 for unknown routes in single-page apps, or use a documented client-side fallback
Rests on 7 claims, 5 of them naming 404; its own words name 404
For an urgent removal, use the Removals tool in Search Console and also return 404 or noindex
Rests on 5 claims, 3 of them naming 404; its own words name 404
Do not answer crawlers with 401, 402 or 403 on pages that should be in Search
Rests on 5 claims, 3 of them naming 404; its own words name 404
Return 404 or 410 for removed and non-existent URLs
Rests on 7 claims, 2 of them naming 404; its own words name 404
Never serve error states, empty results or failed data loads with a 200 status
Rests on 10 claims, 2 of them naming 404
5 more
If facet URLs must be crawlable, use & separators, a fixed filter order and 404 for empty results
Rests on 5 claims, 2 of them naming 404; its own words name 404
Redirect an old URL only to a page with the same intent, and return 404 or 410 when there is none
Rests on 7 claims, 1 of them naming 404; its own words name 404
Plan releases and fixes around Google's processing times, and judge results only after them
Rests on 11 claims, 1 of them naming 404; its own words name 404
Answer planned maintenance and short outages with 503, for a day or two at most
Rests on 11 claims, 1 of them naming 404; its own words name 404
Keep robots.txt reachable on every host: 200 with rules, or 404 if there are none
Rests on 4 claims, 0 of them naming 404; its own words name 404
Connected things 22
Relations
- Is a 4xx structure, no claim needed
Most often named with it
Things named in the same claim, with the number of claims they share.
- Soft 404 8
- 410 7
- noindex 7
- 200 6
- Redirects 6
- Single-page app 5
- JavaScript 4
- 4xx 3
- Googlebot 3
- 403 2
- 429 2
- 503 2
- Crawl budget 2
- Main content 2
- Robots meta tag 2
- Site move 2
Topics that feature it
- Soft 404s 11
- JavaScript SEO pitfalls and fixes 9
- Crawl errors 5
- AI crawlers and agents on your site 3
- How long Google's processes take 3
- Redirects, site moves and alternate names 2
- Robots meta tags and noindex 2
- Canonical selection 1
- Crawl budget 1
- Crawl rate limit (hostload) 1
- Directives and crawl budget 1
- Duplicate content 1
- Faceted navigation 1
- How Google renders pages 1
- Pruning and consolidating content 1