Crawl budget refers to the number of pages a search engine’s crawler will crawl on a given website within a given timeframe. It’s determined by a combination of how much a search engine wants to crawl a site (based on its perceived value and update frequency) and how much the site’s server can handle without being overwhelmed.
Crawl budget becomes a genuine concern primarily for larger websites with thousands of pages, such as e-commerce stores or large publishers, where wasted crawl budget on low-value pages (like faceted navigation URLs or thin duplicate content) can mean important pages get crawled less frequently, or not at all.
Optimizing crawl budget typically involves blocking low-value pages via robots.txt, fixing broken links and redirect chains, and maintaining a clean, efficient site architecture that directs crawlers toward the pages that matter most.
Crawl budget optimization is part of our Technical SEO service, particularly for larger sites.