Knowledge base v2.13.0 · Community edition · data through 2 October 2026
Day 3 · Friday 2 October 2026 · 15:45
Speaker from the author's recording label. Google's estimated processing times for crawling, indexing and serving, based on internal analysis.
Each of Google's processing-time charts marks a minimum, a typical value and an end point on a scale from seconds to one year, with a dotted line for corner cases, and carries the footnote that the times are estimations based on internal analysis.
“Times expressed on the diagram are estimations based on internal analysis.”
Wording checked against the slide or recording
Speaker Gary IllyesEvidence 4 slide photos
Google estimated that discovering a new URL takes about 20 hours on average across all sites it knows of, with a minimum of seconds.
Speaker Gary IllyesEvidence 2 slide photos, transcript
Google's crawl chart says the discovery of a new URL can take weeks or never happen, because the web is huge.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that refreshing (recrawling) a known URL takes about 30 days on average, with a minimum of seconds and an end point of weeks to never.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that processing a sitemap takes about 24 hours on average, with a minimum of minutes.
Speaker Gary IllyesEvidence slide photo, transcript
If a sitemap is useful to Google and the site is of high quality, Google tries to refetch the sitemap within 14 days at most.
Speaker Gary IllyesEvidence slide photo, transcript
Google may never fetch a lower-quality site's sitemap again: once it figures out the site is of lower quality, it no longer wants to fetch the sitemap.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that a robots.txt update is picked up in about 24 hours, with a minimum of seconds and an end point of 25 hours on the slide.
Speaker Gary IllyesEvidence slide photo, transcript
Used byrequirements DEV-MON-10, DEV-SRV-05
Google's crawl chart puts a crawl capacity update at seconds when backing off, typically 4 hours or 1-2 weeks, and 1-3 weeks in recovery.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that a crawl demand update driven by Search takes about 20 hours on average, with a minimum of minutes and an end point of weeks to months.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that rendering takes seconds when done right away and hours when the page waits in the queue, with an end point of days to weeks.
Speaker Gary IllyesEvidence slide photo, transcript
Used byglossary term Render queue
Google estimated that meta annotations, such as robots meta tags, are processed in typically 45 to 90 minutes, with a minimum of 200 seconds (about three minutes) and an end point of 1 to 4 days.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that link annotations are processed in minutes to 1-3 weeks, with an end point of about one year on the slide.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that indexing a document end to end takes about 1.5 hours on average, with a minimum of seconds.
Speaker Gary IllyesEvidence slide photo, transcript
Google defined indexing end to end as the time from a document entering indexing until its critical processes finish and it reaches the serving index, tokenized and ready to be served as a result.
Speaker Gary IllyesEvidence slide photo, transcript
Google's indexing chart says end-to-end indexing can take months or never happen, for reasons of quality.
Speaker Gary IllyesEvidence slide photo
Google estimated that a removal requested by the site owner in Search Console takes effect in about 2 hours on average, from minutes up to 24 hours.
Speaker Gary IllyesEvidence slide photo, transcript
Used byrequirement DEV-IDX-13
Google estimated that a snippet update takes 1-2 days on average and up to several weeks to months, because the page has to be reprocessed to extract the new text.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that updating the image shown with a text result takes 1-2 weeks on average, from a minimum of days up to several weeks to months.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that a title update takes as long as a snippet update: 1-2 days on average, up to several weeks to months.
Speaker Gary IllyesEvidence slide photo, transcript
Used byrequirements DEV-HTM-03, DEV-MON-10
Google estimated that removing a manual action after a reconsideration request takes 1-2 weeks on average, at least 2-3 days, and 4-6 weeks or much longer for dormant sites.
Speaker Gary IllyesEvidence slide photo, transcript
Used byrequirement DEV-SPM-04
Google estimated that a core update affects a site within 2 to 4 weeks from the beginning of its rollout.
Speaker Gary IllyesEvidence slide photo, transcript
Recovery after a core update typically takes 3 to 6 months once the site has put in work to regain traffic, and up to 6 months to a year, until the next core update.
Speaker Gary IllyesEvidence slide photo, transcript
Google estimated that a spam update affects sites within 1-2 days of the rollout, 1-2 weeks for continuous processing (said on stage as 2 weeks on average) and months for batch refreshes.
Speaker Gary IllyesEvidence slide photo, transcript
Google presented its processing times for crawling, indexing and serving as an experiment: estimates from its internal logs, meant to be informative and not something to obsess about, with feedback from the audience invited.
“if you have an engineering mindset, then you can put a number to it.”
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-MON-10
Google said it knows hundreds of trillions of URLs (as of October 2026).
Speaker Gary IllyesEvidence transcript
Google does not crawl all the URLs it knows, because crawling everything would generally waste resources and a much smaller crawl space is enough.
Speaker Gary IllyesEvidence transcript
Google's index drops URLs that have not been recrawled for a very long time, which is the 'never' end of the refresh estimate.
Speaker Gary IllyesEvidence transcript
For news sites in particular, Google said the refresh of a known URL can literally take seconds (the reading 'news sites' is likely but not certain).
Speaker Gary IllyesEvidence transcript
An audience member reported recrawl intervals on one very large, popular client site: the homepage about 5 times a day, first-level pages about every 1.5 days (uncertain reading), and pages clustered as soft 404s every 160 to 190 days, adding that other sites will differ.
From the audienceEvidence transcript
An audience member said that on the large site they observed, recrawl frequency followed the site hierarchy, demand and how well each page is internally linked.
From the audienceEvidence transcript
Google said its service level objective is to refresh robots.txt every 24 hours, as RFC 9309 calls for, though delays happen now and then.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-SRV-05
A site owner can submit robots.txt in Search Console to force Google to refresh it sooner.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-SRV-05
When a site starts serving 500 errors, Google lowers the crawl capacity allocated to the site within about four hours on average.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-SRV-03
Increases in crawl capacity take longer than decreases, within one to three weeks, because Google first needs to know that the higher demand will last.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-SRV-03
A continuously running process recalculates each site's crawl capacity within a month, so a capacity change takes up to about a month at most.
Speaker Gary IllyesEvidence transcript
When Google notices the web getting excited about a few URLs on a site, it allocates more crawl demand to the whole site so it does not miss content useful to future searchers.
Speaker Gary IllyesEvidence transcript
Crawl demand can also come from other Google products, such as Shopping, and the roughly 20-hour estimate covers only demand from Search.
Speaker Gary IllyesEvidence transcript
Google renders pages in two ways: immediately after crawling, or later through a queue-based process that runs elsewhere.
Speaker Gary IllyesEvidence transcript
Content that JavaScript adds to a page is typically seen by Google's indexing system within a few hours, and at worst within weeks.
Speaker Gary IllyesEvidence transcript
Used byrequirements DEV-MON-10, DEV-REN-01
Gary Illyes said Google keeps saying it renders every URL on the internet and that he would say this is not true, though it is what he was told; he went on to say that Google's logs show the rendering queue cleared within weeks.
“we keep saying that we render every single URL on the internet. I would say that that's not true”
Speaker Gary IllyesEvidence slide photo, transcript
Used byrequirement DEV-REN-01
According to Google's logs, everything in the rendering queue gets rendered within weeks.
Speaker Gary IllyesEvidence slide photo, transcript
Used byglossary term Render queue
Meta annotation processing is a critical step of indexing: until a document passes it, Google cannot go on indexing the document.
Speaker Gary IllyesEvidence transcript
Gary Illyes said the 1.5-hour indexing average is probably lowered by selection bias, because so many news sites push out articles.
Speaker Gary IllyesEvidence transcript
After Google processes a page that now returns a 404 or a noindex, it usually removes the page from its serving index within one to three weeks, sometimes much sooner.
Speaker Gary IllyesEvidence transcript
Used byrequirements DEV-IDX-13, DEV-MON-10
A change of canonical URL usually takes one to three weeks to show, though it can happen in seconds because Google allocates enough resources to canonicalization.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-MON-10
Gary Illyes said he had updated Google's documentation about canonicalization changes about a month before the event (around early September 2026).
Speaker Gary IllyesEvidence transcript
Google treats a site move as a complex canonicalization process in which every signal of the old site is recalculated and moved to the new one, and every indexing process has to run.
Speaker Gary IllyesEvidence transcript
A site move can finish within a few weeks for a small site, takes one to three months on average, and in the worst case about a year.
Speaker Gary IllyesEvidence transcript
Used byrequirements DEV-CAN-09, DEV-MON-10
Gary Illyes said a site move can take up to about a year in the worst case, because Google's slowest signal is recalculated only about once a year (some words of this passage are uncertain readings).
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-CAN-09
Google usually picks up structured data updates within hours to one or two weeks, sometimes immediately.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-MON-10
Google may never use structured data from a site it does not trust: once it sees markup it does not trust, it does not touch it.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-SDA-02
Google indexes images in hours to two days on average, and the fastest it can reliably index an image is under 150 seconds.
Speaker Gary IllyesEvidence transcript
Google tries to index images from news sites much faster than usual, which lowers the average image indexing time (the reading 'news sites' is likely but not certain).
Speaker Gary IllyesEvidence transcript
Google's deep analysis of a video, needed to actually understand it, can take months to finish.
Speaker Gary IllyesEvidence transcript
Search Console removals average about 2 hours because Google pushes removals out to its serving system about every 2 hours.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-IDX-13
The image shown with a text result changes only after the image itself is indexed or reindexed and then goes through a further processing step involving embeddings.
Speaker Gary IllyesEvidence transcript
For some sites a manual action is almost never removed, because the owners do not make the effort to actually clean up the site.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-SPM-04
Google's processes depend on each other: crawling happens first, then indexing, then serving, so a change such as a new canonical cannot be processed until the page has been recrawled.
Speaker Gary IllyesEvidence transcript
Used byrequirement DEV-MON-10
Google said it structured the three days of the event as crawling, indexing and serving because that is literally how Search works.
Speaker Gary IllyesEvidence transcript
Google's robots.txt guide says its crawlers update their cached copy of a site's robots.txt every 24 hours, and that the Request a recrawl option in Search Console's robots.txt report refreshes it faster.
Publisher Google Search Central, Google Search Console Help
Used byrequirement DEV-SRV-05
Google's crawl rate guide says that when a significant number of URLs return 500, 503 or 429, Google reduces the site's crawl rate, which starts increasing again automatically once the errors drop; it warns against doing this for longer than 1-2 days.
Publisher Google
Used byrequirement DEV-SRV-03
Google's guide to fixing canonicalization issues says that even after content issues are fixed, Google might hold pages in a duplicate cluster for up to two weeks, and that pages split out faster when they differ clearly and significantly.
Publisher Google Search Central
Used byrequirement DEV-MON-10
Google's site move guide says to keep redirects for as long as possible, generally at least one year, so that Google can transfer all signals to the new URLs.
Publisher Google Search Central
Used byrequirement DEV-CAN-09
Google's site move guide says that, as a general rule, a small to medium-sized site can take a few weeks for most pages to move and larger sites take longer, depending on the number of URLs and server speed.
Publisher Google Search Central
Used byrequirement DEV-CAN-09
Google's Removals tool help says a temporary removal request usually takes up to a day to process and lasts only about six months, so the content must also be removed permanently, for example with a 404 or 410, a noindex or a password.
Publisher Google Search Console Help
Used byrequirement DEV-IDX-13
Google's Manual actions report help says most reconsideration reviews take several days or weeks, and that some, such as link-related requests, may take longer.
Publisher Google Search Console Help
Used byrequirement DEV-SPM-04
Google's title link guide says Google has to recrawl and reprocess a page to notice changes to the sources of its title link, which may take a few days to a few weeks.
Publisher Google Search Central
Used byrequirements DEV-HTM-03, DEV-MON-10
With a typical refresh of about 30 days and deep or soft-404-like pages recrawled only every few months, link important deep pages from strong hub pages and fix pages that look like errors; then check recrawl intervals by site depth in server logs or the Crawl Stats report.
Author Ibrahim Anjro
Plan robots.txt changes a day ahead, because Google may use the old file for about 24 hours; for an urgent change, request a refresh in Search Console instead of waiting.
Author Ibrahim Anjro
Spoken and slide figures for capacity increases differ (one to three weeks, up to a month, versus 1-2 weeks typical and 1-3 weeks in recovery), but the lesson is the same: a burst of 5xx errors cuts crawling within hours and recovery takes weeks, so keep servers stable before launches and migrations.
Author Ibrahim Anjro
Used byrequirement DEV-SRV-03
Put critical content, links and meta tags in the server HTML: content that needs JavaScript reaches indexing only after rendering, usually within hours but sometimes weeks, and even Google's own speaker doubted that every URL gets rendered.
Author Ibrahim Anjro
By Google's own averages, getting found and crawled takes far longer than indexing (about 20 hours to discover a URL and 30 days to refresh one, against 1.5 hours to index), so for faster results work on discovery: internal links from often-crawled pages and accurate sitemaps.
Author Ibrahim Anjro
Keep migration redirects in place for at least a year and judge a site move after one to three months, not days: Google's speaker said its slowest signal needs about a year to be recalculated (a partly uncertain passage of the recording), and Google's site move guide says to keep redirects generally at least one year.
Author Ibrahim Anjro
Used byrequirement DEV-CAN-09
To hide a URL from Google Search urgently, use the Removals tool in Search Console (about 2 hours) and also add a noindex or a 404, which takes one to three weeks to drop the page from the index.
Author Ibrahim Anjro
Used byrequirement DEV-IDX-13
After changing a title or meta description, wait at least a few days before judging the result in Search; if nothing changes after several weeks, check that the page was recrawled, since snippets and titles are only re-extracted from a reprocessed page.
Author Ibrahim Anjro
Set expectations for recovery work: a manual action can lift within weeks of a real clean-up, but recovery from a core update typically takes 3 to 6 months and may need the next core update, so do not judge quality work within the first weeks.
Author Ibrahim Anjro
The canonicalization documentation update mentioned on stage appears in Google's documentation changelog on 10 July 2026 (re-evaluation time, page last updated 21 August 2026), earlier than 'about a month' before the event; its 'up to two weeks' sits at the short end of the one to three weeks said on stage.
Author Ibrahim Anjro
The stage doubt that every URL gets rendered sits beside Google's JavaScript guide, which says every page with a 200 status is queued for rendering unless a robots rule blocks indexing: queued is not the same as rendered, so do not rely on rendering for critical content.
Author Ibrahim Anjro
Used byrequirement DEV-REN-01
Google's ranking systems guide speaks of hundreds of billions of pages in its Search index; by the author's arithmetic, against the hundreds of trillions of URLs Google said it knows, only around one known URL in a thousand is indexed, if both figures are taken at face value.
Author Ibrahim Anjro
Google said it knows hundreds of trillions of URLs (as of October 2026).
Google said there are trillions of URLs on the internet, or even more, that even Google cannot tell how many exist, and that some may never be discovered.
For news sites in particular, Google said the refresh of a known URL can literally take seconds (the reading 'news sites' is likely but not certain).
Google estimated that processing a sitemap takes about 24 hours on average, with a minimum of minutes.
An XML sitemap, a format from around 2005, gives Google, other search engines and potentially AI systems a list of the URLs a site wants crawled; Google called it nothing fancy but said it still uses sitemaps.
If a sitemap is useful to Google and the site is of high quality, Google tries to refetch the sitemap within 14 days at most.
An XML sitemap, a format from around 2005, gives Google, other search engines and potentially AI systems a list of the URLs a site wants crawled; Google called it nothing fancy but said it still uses sitemaps.
Google may never fetch a lower-quality site's sitemap again: once it figures out the site is of lower quality, it no longer wants to fetch the sitemap.
Google's crawl scheduler very likely deprioritises a URL when the URL or its site is known to be historically spammy.
Google estimated that a robots.txt update is picked up in about 24 hours, with a minimum of seconds and an end point of 25 hours on the slide.
Google generally caches robots.txt for up to 24 hours and reads only the first 500 KiB of the file.
Google said its service level objective is to refresh robots.txt every 24 hours, as RFC 9309 calls for, though delays happen now and then.
Google generally caches robots.txt for up to 24 hours and reads only the first 500 KiB of the file.
A site owner can submit robots.txt in Search Console to force Google to refresh it sooner.
Search Console's robots.txt report shows the robots.txt files Google found for the top 20 hosts of a Domain property, with their fetch status, warnings and errors; to test whether a specific URL is blocked, Google's help page points to the URL Inspection tool and to Google's open-source robots.txt library.
Google's crawl chart puts a crawl capacity update at seconds when backing off, typically 4 hours or 1-2 weeks, and 1-3 weeks in recovery.
When a site starts serving 500 errors, Google lowers the crawl capacity allocated to the site within about four hours on average.
Google treats network timeouts, connection resets and DNS errors like 5xx server errors: crawling slows down immediately, and already indexed URLs that stay unreachable are removed from Google's index within days.
When a site starts serving 500 errors, Google lowers the crawl capacity allocated to the site within about four hours on average.
Google slows crawling when a site returns 5xx errors, because a 5xx usually means the server, and often the whole site, cannot serve requests, and Google does not want to break the site.
When a site starts serving 500 errors, Google lowers the crawl capacity allocated to the site within about four hours on average.
When Google notices the web getting excited about a few URLs on a site, it allocates more crawl demand to the whole site so it does not miss content useful to future searchers.
Crawl demand is driven by the quality of the site, the change frequency of its URLs and their popularity on the internet.
When Google notices the web getting excited about a few URLs on a site, it allocates more crawl demand to the whole site so it does not miss content useful to future searchers.
The quality that drives crawl demand is the quality of the site as a whole, not the quality of an individual page.
When Google notices the web getting excited about a few URLs on a site, it allocates more crawl demand to the whole site so it does not miss content useful to future searchers.
Crawl demand can also come from other Google products, such as Shopping, and the roughly 20-hour estimate covers only demand from Search.
Googlebot as a single standalone crawler is a historical idea: Google crawls through a centralised crawling infrastructure, so a request from a Google user agent in server logs is a request routed through that shared platform.
Google renders pages in two ways: immediately after crawling, or later through a queue-based process that runs elsewhere.
After processing, an indexable page is placed in Google's render queue to wait for rendering.
Content that JavaScript adds to a page is typically seen by Google's indexing system within a few hours, and at worst within weeks.
Google's JavaScript SEO basics guide says a page may wait in the render queue for a few seconds but that it can take longer, and it gives no upper limit.
Gary Illyes said Google keeps saying it renders every URL on the internet and that he would say this is not true, though it is what he was told; he went on to say that Google's logs show the rendering queue cleared within weeks.
Google renders nearly all of the web by replicating what a browser does, using a real browser's rendering engine.
Google treats a site move as a complex canonicalization process in which every signal of the old site is recalculated and moved to the new one, and every indexing process has to run.
Google treats a site migration as deduplication across sites, in which the site owner says the old and the new domain are the same and that Google should pick the new domain, so deduplication also helps Google handle migrations.
Google may never use structured data from a site it does not trust: once it sees markup it does not trust, it does not touch it.
Structured data that is not relevant to the page's content can be treated as abusive: Google's filters make it ineffective, and egregious cases can lead to a manual action.
Recovery after a core update typically takes 3 to 6 months once the site has put in work to regain traffic, and up to 6 months to a year, until the next core update.
After a core update a site can win back some of its rankings by improving the site as a whole.
Google estimated that a spam update affects sites within 1-2 days of the rollout, 1-2 weeks for continuous processing (said on stage as 2 weeks on average) and months for batch refreshes.
When spammers find loopholes that Google's systems miss, Google releases spam updates that change its systems to catch them.
Google's robots.txt guide says its crawlers update their cached copy of a site's robots.txt every 24 hours, and that the Request a recrawl option in Search Console's robots.txt report refreshes it faster.
Search Console's robots.txt report shows the robots.txt file as Google last fetched it, with a version history and the errors and successes of each fetch.
The stage doubt that every URL gets rendered sits beside Google's JavaScript guide, which says every page with a 200 status is queued for rendering unless a robots rule blocks indexing: queued is not the same as rendered, so do not rely on rendering for critical content.
Google's JavaScript SEO guide says Googlebot sends every page with a 200 HTTP status code to the rendering queue, whether or not it contains JavaScript, unless a robots meta tag or header tells Google not to index it, and Google uses the rendered HTML to index the page.
Crawl demand can also come from other Google products, such as Shopping, and the roughly 20-hour estimate covers only demand from Search.