Term
Crawl Budget
Crawl budget is the number of URLs Googlebot crawls on a website within a given time frame — determined by server load and the perceived popularity of the content.
Crawl budget — explained in more detail
Crawl budget combines two components: the crawl-rate limit (how many requests Googlebot is willing to send the server, derived from response times and error rates) and crawl demand (how worthwhile Google considers re-fetching the content — driven by popularity and change frequency). Together they determine how many URLs Googlebot actually fetches per time window on a given site.
Example / context
Crawl budget only becomes a concern on large sites — a rough rule of thumb is from roughly 10,000 URLs upwards, or sites with very frequent updates (news, large shops with dynamic filters). For most small and mid-sized sites the budget is effectively unlimited; new or changed content is crawled in good time without any tuning. If you do want to optimize: reduce duplicate content, block parameterized URLs in robots.txt, maintain clear internal linking, keep server response times fast.
Entdecke mehr
301 Redirect
A 301 redirect is an HTTP status code permanently redirecting one URL to another — the standard for URL changes, domain moves and site migrations.
LexikonTechnical SEO — what Google actually has to crawl, render and index
How crawl budget, robots, sitemap, JS rendering, indexing, canonical and Core Web Vitals fit together — the full arc for production sites.
NewsGoogle Search Central Live Toronto 2026 — what Google officially said about the future of search
Information Gain as the new guideline, Google-Extended clarified, AI Mode with 93% zero-click. The key statements from Toronto.