Term
Crawl Rate
The speed at which a search-engine crawler issues requests to a site — measured in requests per second or per day and automatically adjusted to server response time.
Crawl Rate — explained in more detail
Googlebot and friends regulate crawl rate themselves, based on server response: fast, error-free answers → higher rate; frequent 5xx errors or long response times → rate automatically drops. The goal is “crawl as much as possible without overloading the site”.
In Search Console, crawl rate can no longer be manually raised (the old setting was removed in 2024) — instead, in case of overload, you can submit a throttling request via the crawl-rate-reduce form. Bing Webmaster Tools still offers a time-based throttling profile.
Example / practical use
A small site with 200 URLs and stable 200 responses typically gets fully crawled every few days. A site with frequent 5xx or timeouts shows a falling curve in the crawl-stats report — a classic early warning for server issues.
Distinction from related terms
Crawl rate = speed of requests. Crawl demand = how many URLs the bot actually wants to crawl (popularity, freshness). Together they form the crawl budget — the practical output for large sites.
Entdecke mehr
301 Redirect
A 301 redirect is an HTTP status code permanently redirecting one URL to another — the standard for URL changes, domain moves and site migrations.
LexikonTechnical SEO — what Google actually has to crawl, render and index
How crawl budget, robots, sitemap, JS rendering, indexing, canonical and Core Web Vitals fit together — the full arc for production sites.
NewsGoogle Search Central Live Toronto 2026 — what Google officially said about the future of search
Information Gain as the new guideline, Google-Extended clarified, AI Mode with 93% zero-click. The key statements from Toronto.