WordPress Plugin Crawl Budget Killers: How to Fix It
Here’s an uncomfortable truth most WordPress site owners don’t want to hear: your crawl budget problems probably aren’t your fault—but they’re absolutely your responsibility. Google’s internal 2025 year-end crawl report reveals that the plugins you trust are silently hemorrhaging your site’s crawl resources, and the fixes aren’t coming fast enough.
Gary Illyes from Google’s Search Relations team dropped this bombshell on the Search Off the Record podcast, exposing how his crawl team has been filing bugs directly against WordPress plugins. As Search Engine Journal reports, WooCommerce fixed their add-to-cart parameter issue quickly, but other plugins haven’t responded at all. This is the story nobody in the WordPress ecosystem wants to talk about—and the actionable steps you need to take right now.
What Google’s Data Actually Shows
According to Search Engine Land’s coverage, the biggest crawl issues were faceted navigation and action parameters, which accounted for about 75% of the problems. Let that sink in. Three-quarters of every severe crawl problem Google flagged last year comes from just two technical categories—and both are often baked into the plugins running your site.
The Breakdown of Crawl Budget Waste
Stan Ventures’ analysis of Google’s data breaks down the crawl waste categories:
The single biggest drain on crawl budget remains faceted navigation. Illyes revealed that roughly half of all severe crawling reports come from sites allowing users to filter content without controlling the resulting URLs.
A surprising 25% of crawl budget waste comes from action parameters—those innocent-looking URL additions like ?add-to-cart=true.
Session IDs still account for 10% of crawl issues. If your site appends a unique ID to the URL for every visitor, Googlebot views every single session as a unique page.
The final category involves infinite spaces, often caused by calendar widgets or event plugins that generate endless date-based URLs.
What Most People Get Wrong About Crawl Budget
Here’s the contrarian take that might save you hours of unnecessary optimization: most sites don’t have a crawl budget problem. Google’s official documentation states that if your site doesn’t have a large number of pages that change rapidly, or if your pages seem to be crawled the same day they are published, you don’t need to worry. This guide is intended primarily for large sites with 1 million+ unique pages, or medium sites with 10,000+ pages and rapidly changing content.
However, WooCommerce and ecommerce sites are the exception. Even “medium-sized” stores can generate hundreds of thousands of URL variations through filter combinations, making them prime candidates for crawl waste.
How Action Parameters Destroy Your Crawl Budget
The problem with action parameters is deceptively simple. As Search Engine Journal explains, each parameter creates what appears to be a new URL by adding text like ?add_to_cart=true. Parameters can stack, doubling or tripling the crawlable URL space on a site.
Think about it this way: your product page at /blue-widget/ is one URL. But add a cart parameter, and it becomes /blue-widget/?add-to-cart=true. Now combine that with session tracking, wishlist functionality, and currency selection, and you’ve created a URL multiplication nightmare.
“The things that Googlebot tends not to do is shop around on the internet. It will not buy your weirdo hoodie,” Illyes joked during the podcast. As Stan Ventures notes, since Googlebot doesn’t make purchases, every millisecond it spends crawling an ?add-to-cart URL is a direct waste of budget that could have been used to index a new blog post or product page.
Real-World Example: The Filter Explosion
One UK-based WooCommerce store hit a major issue with Google over-indexing filter-based URLs. According to a WordPress.org support thread documenting the case, the site experienced about 6,000 extra indexed pages from filter URLs—causing CPU usage spikes from 40,000 seconds/day to 400,000+ seconds/day.
That’s a 10x increase in server resource consumption from URLs that provide zero business value.
The WooCommerce Fix
Search Engine Journal’s reporting reveals that Google’s crawl team filed an issue against WooCommerce after identifying its add-to-cart URL parameters as a top source of crawl waste.
What made WooCommerce’s response exemplary wasn’t just speed—it was accountability. Illyes described how his team would “try to dig into where these are coming from” and identify that “these action parameters are coming from WordPress plugins.” They would then file issues against plugins that have open-source repositories.
WooCommerce responded and shipped a fix quickly. Illyes noted the turnaround was fast, but other plugin developers with similar issues haven’t responded.
Plugins That Still Haven’t Fixed Their Issues
Not every plugin developer has been as responsive. According to Search Engine Journal, an issue filed against a separate action-parameter plugin is still sitting unclaimed. And a commercial calendar plugin that generates infinite URL paths reportedly ignored Google’s direct outreach.
This creates a problematic pattern: plugin developers prioritize features over SEO hygiene, leaving site owners to clean up the mess.
Why Documentation Alone Doesn’t Solve This Problem
Here’s the uncomfortable reality Illyes essentially admitted: this is the same URL parameter problem he warned about before and continued flagging. Google then formalized its faceted navigation guidelines into official documentation. As Search Engine Journal notes, the data shows those warnings and documentation updates didn’t solve the problem.
Translation: Google can publish all the best practices they want, but if the code creating the problem is embedded in plugins, most site owners will never even know they have an issue.
The Hidden Catch-22 of Crawl Discovery
According to Search Engine Journal’s coverage, Illyes said Googlebot can’t determine whether a URL space is useful “unless it crawled a large chunk of that URL space.” This creates a frustrating paradox: Googlebot must waste your crawl budget to discover that URLs are wasteful.
Stan Ventures reports that Illyes warned when Googlebot discovers vast new URL patterns, it goes into discovery mode. “Googlebot will want to crawl all those URLs to make a decision whether it should crawl or should not crawl those URLs.” During this phase, the bot acts “like a madman,” burning through your crawl capacity.
By the time you notice server strain or indexing delays, the damage is already happening.
5 Practical Tips to Fix It
1. Robots.txt Blocking (The Primary Defense)
Google’s official faceted navigation documentation recommends using robots.txt to disallow crawling of faceted navigation URLs. Oftentimes there’s no good reason to allow crawling of filtered items, as it consumes server resources for no or negligible benefit.
Example robots.txt rules for WooCommerce:
Disallow: /*?add-to-cart=*
Disallow: /*?add_to_wishlist=*
Disallow: /*?filter_*
Disallow: /*?orderby=*
2. Switch Action Parameters to POST Requests
Stan Ventures recommends working with developers to ensure actions like “Add to Cart” use HTTP POST requests (which bots generally don’t follow) rather than GET requests.
This is the most elegant solution—it prevents the URLs from ever being generated in a crawlable form.
3. The noindex Trap to Avoid
Verkeer’s crawl budget guide specifically advises that if you want to save crawl budget, don’t rely on just noindex! A noindexed page still gets crawled (Google finds the noindex after fetching the page, meaning the crawl budget was already spent). For pages that are truly junk for search, it’s often better to disallow them via robots.txt so Google never fetches them at all.
4. Audit Your Plugin-Generated URLs
Run a crawl using Screaming Frog or Sitebulb and filter for URLs containing common action parameters: ?add, ?action, ?sid, ?session, ?filter, ?sort. If you see thousands of these variations, you have a problem.
5. Leverage SEO Plugins for Parameter Management
According to WPBeginner, the easiest way to optimize crawl budget is by using a plugin like All in One SEO. Its built-in Crawl Cleanup feature allows you to easily stop search engines from crawling low-value pages, such as pages with certain query parameters.
The WordPress.org support discussion notes that Yoast SEO also has a feature to remove unregistered URL parameters. However, because it affects other store features, it’s not recommended for ecommerce sites.
Enterprise WordPress Sites: The 40-60% Waste Problem
Large WordPress sites frequently hemorrhage crawl budget through inefficient architecture. OnwardSEO’s analysis reveals that sites exceeding 50,000 pages typically waste 40-60% of their allocated crawl budget on low-value URLs through pagination chains, faceted navigation, duplicate content, and plugin-generated URLs.
Attachment Pages: The Silent Killer
According to OnwardSEO’s WordPress audit guide, the most significant waste occurs through attachment pages, which WordPress generates automatically for every media file upload. A typical enterprise WordPress site with 50,000 uploaded images creates 50,000 indexable attachment URLs that provide zero search value.
Quick fix: Use Yoast or RankMath to redirect attachment pages to parent posts, or block them in robots.txt with Disallow: /*attachment_id=*.
Should Google Just Be Smarter?
There’s a legitimate counter-argument here: why can’t Google simply recognize that /product/?add-to-cart=true is the same page as /product/? Google has stated they can detect many parameter-based duplicates, but the sheer scale of WordPress installations and plugin variations makes perfect detection impossible.
Search Engine Journal reports that Illyes has also challenged conventional wisdom with a potentially controversial claim: “It’s not crawling that is eating up the resources, it’s indexing and potentially serving or what you are doing with the data.”
This reframes the crawl budget debate somewhat—but it doesn’t change the fact that wasted crawling leads to wasted indexing attempts, which still consumes resources and delays discovery of your important content.
Google’s Plugin Bug-Filing Strategy
As Search Engine Journal notes, Google filing bugs against open-source plugins could help reduce crawl waste at the source. This represents a shift from documentation-based guidance to direct intervention.
But here’s what concerns me: this approach only works for open-source plugins with active maintainers. Commercial plugins, abandoned projects, and closed-source solutions remain beyond Google’s reach. Site owners using these tools are on their own.
The AI Traffic Problem on the Horizon
Search Engine Journal also reports that Illyes warned that AI agents and “AI shenanigans” will be significant sources of new web traffic. According to Apple Podcasts’ listing for Search Off the Record, the rising impact of automated traffic from new products and AI agents is creating growing internet congestion.
If your site already has crawl budget issues from plugins, adding AI crawler traffic will compound the problem significantly.
FAQ: WordPress Crawl Budget Issues
Does every WordPress site need to worry about crawl budget?
According to WPBeginner, while crawl budget is a factor for every website, it becomes a more significant concern for larger sites with thousands of pages. Even smaller sites benefit from good crawl optimization—it ensures search engines can efficiently find all content without wasting resources.
What’s the fastest way to identify crawl waste on my site?
Check Google Search Console’s Crawl Stats report under Settings. Look at which URL patterns Google is crawling most frequently. If you see parameter-heavy URLs dominating, you have an action-parameter or faceted navigation problem.
Will blocking URLs in robots.txt give me more crawl budget for other pages?
LinkGraph’s crawl budget guide explains that Google explicitly warns against using robots.txt to reallocate crawl budget. “Google won’t shift this newly available crawl budget to other pages unless Google is already hitting your site’s serving limit.”
Should I use canonical tags instead of robots.txt?
Canonical tags help with duplicate content but don’t prevent crawling. The page still gets fetched before Google sees the canonical directive. For pure crawl budget optimization, robots.txt blocking is more effective.
Which WooCommerce settings cause the most crawl waste?
According to OnwardSEO’s WooCommerce audit guide, WooCommerce sites require consideration for cart URLs, user account pages, checkout processes, and API endpoints that may be publicly accessible but provide zero search value.
The Bottom Line
As Search Engine Journal summarizes, the crawl waste is often baked into the plugin layer. That creates a real bind for websites with ecommerce plugins. Your crawl problems may not be your fault, but they’re still your responsibility to manage.
The WooCommerce fix shows that responsible plugin developers can solve these issues quickly. But until the entire WordPress plugin ecosystem takes SEO hygiene seriously, site owners need to:
- Audit their URL structures regularly
- Implement robots.txt blocking proactively
- Monitor Google Search Console crawl stats monthly
- Push plugin developers to adopt POST requests for action parameters
Don’t wait for symptoms. Google consistently recommends blocking parameter URLs proactively rather than waiting for symptoms to appear.
Your crawl budget is finite. Stop letting plugins waste it on URLs that will never convert a single customer.
