How CMS Platforms Are Secretly Controlling Your Technical SEO
Most SEOs believe they’re in control. They optimize title tags, build links, and tweak meta descriptions—convinced their efforts move the needle. But here’s the uncomfortable truth from the Web Almanac 2025: your CMS platform is probably shaping your technical SEO far more than any individual optimization you’ll ever make.
Chris Green, an industry expert with over 15 years of experience and one of the authors of the Web Almanac’s SEO chapter, recently shared insights that challenge some fundamental assumptions about where SEO influence actually lies—and why the emerging AI crawler landscape is creating a strategic dilemma most teams aren’t prepared to handle.
Why Your SEO Impact Is Smaller Than You Think
According to the Web Almanac’s CMS chapter, WordPress remains the dominant CMS in 2025, powering roughly 64% of CMS-driven sites—though its growth has slowed to under one percentage point year over year. This isn’t just a market share statistic—it’s a revelation about where technical SEO decisions are actually being made.
Green’s analysis surfaced an insight many practitioners will find humbling. The same HTTP Archive research shows that Webflow and Wix achieve perfect Lighthouse SEO scores, while WordPress, Duda, and Joomla remain around 92. The small year-over-year changes suggest that basic SEO best practices are now widely baked into modern CMS platforms.
Translation? Platform defaults now handle most of the technical SEO heavy lifting. When Wix automatically implements perfect Lighthouse SEO scores out of the box, your manual optimizations become incremental at best.
What Most People Get Wrong About CMS Influence
The conventional wisdom is that WordPress dominates because it’s more “SEO-friendly.” The data tells a different story. Search Engine Journal’s CMS analysis reveals WordPress’s market share shows a sustained decline from its peak of 65.2% in 2022, now back to 2018 levels prior to the pandemic boom—a trend that could continue as easier-to-use platforms gain ground.
Here’s the contrarian view Green’s research supports: WordPress isn’t winning on SEO merit anymore—it’s winning on inertia. Meanwhile, industry tracking data shows Wix’s share climbed with 32.6% growth from 2024 to 2025, demonstrating that easy-to-use builders are capturing a large segment of the market once dominated by WordPress.
The strategic implication? If you want industry-level SEO impact, stop optimizing individual sites and start influencing platform providers. As Green put it, if you’re determined to have an impact beyond your specific clients, “you need to be nudging WordPress or Wix or Shopify or some of the big software providers within those ecosystems.”
Bot Management Is No Longer Binary
For two decades, robots.txt was simple: allow Googlebot, block the scrapers you don’t want. That era is over.
The Register reports that approximately 5.6 million websites have now added OpenAI’s GPTBot to the disallow list in their robots.txt file, up from about 3.3 million at the start of July 2025—an increase of almost 70 percent. And Anthropic’s ClaudeBot is also increasingly wearing out its welcome, now blocked at about 5.8 million websites, up from 3.2 million in early July.
The Strategic Dilemma No One’s Talking About
Green highlights a conflict brewing in most organizations: engineering teams want to reduce server costs by blocking AI crawlers, while editorial teams worry about content being scraped. But there’s a third constituency being ignored—the visibility strategists who understand that blocking certain bots could devastate AI search presence.
The nuance matters because different crawlers serve different purposes. BuzzStream’s research on publisher bot blocking explains that blocking training data prevents a site’s content from being used to train AI models, while blocking retrieval bots prevents the site from appearing when an AI tool attempts to retrieve live data for grounding or citations.
Raptive’s technical documentation clarifies that Google-Extended—the user agent Google uses to access content to train Gemini and other AI products—doesn’t affect your site’s search rankings or inclusion in AI Overviews, according to both Google documentation and direct confirmation from the Google Search team.
But here’s where it gets messy: The Register’s analysis found that across all AI bots, 13.26 percent of requests ignored robots.txt directives in Q2 2025, up from 3.3 percent in Q4 2024. The rules-based approach to bot management is increasingly unreliable.
The Hidden Cost of Bot Blocking
BuzzStream’s study reveals an important nuance: not all bots block you from appearing in the same way. Sites blocking training data can still appear in citations coming from live retrievals. Sites blocking live retrievals can still show up as citations when pulled from the knowledge base.
This creates a strategic paradox: blocking AI crawlers for content protection might not actually protect you from AI systems citing your content, while it definitely removes your ability to influence how you appear in AI-generated answers.
The llms.txt Reality Check: 2% Adoption, 0% Platform Commitment
The Web Almanac found approximately 2% adoption for the proposed llms.txt standard—a number Green admitted was higher than expected. But context matters enormously here.
Analysis from llms-txt.io shows the adoption pattern is unmistakable: llms.txt has achieved rapid penetration in developer tools, AI companies, technical documentation, and SaaS platforms, while remaining completely absent from the mainstream web.
More critically, while over 784 websites have implemented llms.txt files—creating a thriving grassroots movement among developer documentation sites—the same research confirms that zero major LLM providers have officially committed to using them. Google has explicitly rejected the standard, comparing it to the discredited keywords meta tag.
Why SEO Tools Are Driving Adoption, Not Strategy
Much of the llms.txt growth appears to be driven by SEO plugins adding the feature, not by strategic decisions. This creates a classic case of premature optimization: sites implementing a standard that no major platform has committed to supporting.
Green’s position is clear: Google has repeatedly stated they don’t need llms.txt because “they do have crawling and rendering nailed.” The real question is whether OpenAI will adopt it—and given their current challenges, establishing new standards probably isn’t a priority.
Big Cloudy’s comprehensive guide notes that over 780+ websites have added llms.txt—including companies everyone respects like Cloudflare, Vercel, and Coinbase. But when Google’s engineers were asked directly, they publicly dismissed it.
AI Overviews Are Rewriting Search Economics
While the Web Almanac focused on technical SEO metrics, the context around AI search has shifted dramatically. The Digital Bloom’s 2025 traffic crisis report found that searches displaying AI Overviews saw click-through rates plummet to just 8%, compared to 15% for traditional search results without AI summaries—a 47% reduction in clicks when AI Overviews are present.
Dataslayer’s analysis of Seer Interactive’s September 2025 study reveals even starker numbers: organic CTR plummeted 61% (from 1.76% to 0.61%) for queries with AI Overviews, while paid CTR crashed 68% (from 19.7% to 6.34%).
The Industry-Specific Devastation
Research on AI Overviews’ traffic impact shows the effects vary significantly across industries: fashion, travel, DIY, and cooking websites have experienced declines in traffic, with some reporting up to a 70% drop.
Press Gazette reports that Google search traffic to publishers declined globally by a third in the year to November. Referrals to more than 2,500 publisher websites from Google Discover were down 21% year on year.
Search Engine Journal’s coverage of Similarweb data shows zero-click searches increased from 56% to 69% between May 2024 and May 2025.
A Glimmer of Nuance in the Data
Before panic sets in, consider this: Search Engine Land’s data analysis found that zero-click behavior defied expectations. Surprisingly, click-through rates for keywords with AI Overviews have steadily risen since January. AI Overviews don’t automatically reduce clicks and may even encourage them.
There’s also a survival path: Dataslayer’s research shows brands cited in AI Overviews earn 35% more organic clicks and 91% more paid clicks. The game has shifted from ranking to being cited.
AI Agents Won’t Replace SEOs—But They’ll Expose Bad Ones
Green offered perhaps his most important insight for practitioners worried about job security: AI agents are not replacements for expertise—they’re accelerators of process. The distinction matters.
His estimate that early-stage agents could automate 60-80% of SEO work sounds alarming until you understand what that work actually is: repetitive data gathering, pattern recognition, and templated analysis. The remaining 20% requiring human judgment? That’s where the real value lives.
But here’s the warning most practitioners need to hear: “You need to understand the process that you’re trying to optimize. If the process isn’t very good, you’ve just created a machine to produce mediocrity at scale, which frankly doesn’t help anyone.”
The Process Quality Test
Ask yourself: if you automated your current SEO workflow tomorrow, would it produce excellence or merely scale your existing inefficiencies? Most honest practitioners would admit the latter.
The opportunity isn’t in resisting automation—it’s in using the freed capacity to do work that actually requires human intelligence: business context, strategic judgment, and the ability to synthesize insights across domains that AI systems still struggle to connect.
What the Web Almanac Actually Reveals About Technical SEO in 2026
The Web Almanac’s SEO chapter states that while many SEO metrics have stabilized across the web, the context surrounding them is changing rapidly. The rise of AI crawlers, emergence of llms.txt, and a growing emphasis on machine readability suggest that optimization is no longer only about being found by bots, but about being understood by them.
The technical fundamentals still matter. The same research shows title tags are present on 98.6% of desktop pages, while meta descriptions appear on only 67.7%. Title tag adoption remains nearly universal, climbing slightly in 2025 to 98.62%—reflecting the element’s enduring role in SEO strategy despite widespread title rewrites in search results.
But the game is evolving beyond traditional metrics. The Web Almanac’s CMS analysis concludes that in a world of evolving search and discovery systems, structured data acts as a stabilizing signal. CMS platforms that validate schema, limit redundancy, and integrate structured data into core templates are more likely to produce durable, machine-friendly content.
What to Do With These Insights
For individual SEOs: Stop optimizing in isolation. Understand how your CMS handles technical SEO by default, and focus your manual efforts on the gaps that actually matter—not the basics your platform already handles.
For SEO managers: Develop an AI crawler strategy before your engineering team decides for you. Create a matrix mapping each major AI bot to its function (training vs. retrieval) and align that with business objectives around AI visibility.
For enterprise teams: Consider your CMS choice as a technical SEO decision, not just a content management decision. The performance and SEO characteristics baked into platforms like Shopify or Webflow might outweigh the flexibility advantages of WordPress for certain use cases.
For agency leaders: Start building process documentation robust enough to survive automation. The agencies that thrive will be those whose human practitioners focus on strategic judgment while automated systems handle execution.
Everything Becomes Complex
The web in 2026 isn’t simpler—it’s more complex than ever. AI crawlers, zero-click searches, platform-level SEO decisions, and rapidly evolving standards mean that SEO practitioners must become systems thinkers, not just tacticians.
Green’s assessment rings true: “The web is really messy. Really messy.” But in that mess lies opportunity. Sites that implement technical standards correctly when most sites lag behind can achieve meaningful differentiation. Practitioners who understand multiple systems—search engines, AI platforms, CMS ecosystems—will be indispensable when those who only understand Google rankings become obsolete.
The SEOs who succeed won’t be those who resist complexity or wait for clarity. They’ll be the ones who engage with uncertainty, build adaptable processes, and recognize that influence now happens across multiple systems simultaneously.
Frequently Asked Questions
What is llms.txt and should I implement it?
Llms.txt is a proposed standard for communicating with AI systems, similar to robots.txt. Currently, approximately 2% of sites have adopted it, but no major AI provider has committed to using it. Google has explicitly rejected the standard. Implementation is low-risk but also provides no proven benefit as of early 2026.
How do AI Overviews affect my SEO traffic?
Research shows AI Overviews can reduce click-through rates by 47-61% when they appear. However, sites cited within AI Overviews see 35% higher organic clicks. The impact varies significantly by industry, with informational content most affected.
Should I block AI crawlers in robots.txt?
This is a strategic decision that depends on your business objectives. Blocking training bots like GPTBot prevents your content from training AI models but may not prevent you from appearing in AI search results. Blocking retrieval bots reduces your visibility in AI-generated answers. Consider the tradeoffs carefully before implementing blanket blocks.
Does my CMS choice affect SEO performance?
Yes, significantly. Web Almanac 2025 data shows CMS platforms have largely standardized basic SEO practices, with most achieving Lighthouse SEO scores between 92-100. Wix and Webflow score perfect 100s, while WordPress averages 92. Platform-level decisions increasingly shape technical SEO more than individual optimizations.
How much of SEO work can be automated with AI?
Industry experts estimate 60-80% of repetitive SEO tasks (data gathering, pattern recognition, reporting) can be automated with current AI tools. The remaining 20% requiring human judgment—strategic planning, business context, and client communication—represents where practitioners can differentiate themselves.
