Bing AI Citation Tracking, Hidden HTTP Homepages & Googlebot’s 2 MB Crawl Limit: What SEOs Must Do Now
The Visibility Gap in AI Search Just Got Smaller — But Only on One Platform
For months, SEO professionals have been flying blind. AI-powered search features cite your content, drive (or suppress) clicks, and reshape your visibility — all without giving you reliable measurement tools.
This week, that changed — partially. Microsoft rolled out an AI citation dashboard in Bing Webmaster Tools, Google’s John Mueller surfaced a sneaky HTTP homepage bug that breaks site names in search results, and new HTTP Archive data confirmed that Googlebot’s recently clarified 2 MB crawl limit is a non-issue for the vast majority of websites.
Each of these developments addresses a gap practitioners have been struggling with. But as we’ll see, each solution also opens up new questions that the SEO community hasn’t fully answered yet.
Bing Webmaster Tools Launches AI Citation Dashboard
Microsoft has introduced a dedicated AI Performance dashboard in Bing Webmaster Tools, giving publishers their first structured view of how often and where their content appears in Copilot and AI-generated answers. The feature is now in public preview.
The dashboard tracks total citations, average cited pages per day, page-level citation activity, and — critically — what Microsoft calls “grounding queries.” These are the specific phrases the AI used when retrieving your content to build its response. Think of grounding queries as the AI equivalent of traditional search queries, except they reflect what the model is looking for rather than what a human typed.
Why This Is a Bigger Deal Than Most People Realize
Here’s what the SEO conversation is largely missing: this isn’t just a new report. It’s the first time any major search engine has given publishers a dedicated, AI-specific analytics layer.
Google includes AI Overviews and AI Mode activity inside Search Console’s existing Performance reports but doesn’t break them out into a separate citation-level dashboard. AI Overviews also collapse all linked pages into a single position value, which severely limits your ability to understand how individual pages perform in AI answers.
Bing’s approach is architecturally different. By isolating AI citation data into its own reporting surface, Microsoft is implicitly arguing that AI visibility is a distinct channel — not just a subset of traditional search performance. That framing matters, because it shapes how you allocate resources.
The Counter-Argument: Does Bing’s Market Share Make This Irrelevant?
Let’s address the elephant in the room. Bing holds roughly 3–4% of global search market share. So should you care?
Yes — but not necessarily for Bing traffic alone. There are three reasons this dashboard matters even if your Bing traffic is marginal. First, Copilot is embedded in Windows, Edge, Microsoft 365, and enterprise workflows, meaning its AI answer surface reaches far more users than Bing.com search alone. Second, the grounding query data provides insight into how large language models in general retrieve and cite content — patterns that likely carry over to Google’s Gemini-powered features and even ChatGPT’s browsing capabilities. Third, this sets a competitive precedent. Google has historically matched or surpassed features that gain practitioner adoption on rival platforms. Search Console’s structured data reporting, for example, expanded significantly after Bing Webmaster Tools added richer markup diagnostics.
Wil Reynolds of Seer Interactive noted on X that the grounding queries data is the real prize, though the work of understanding what it means operationally is just beginning.
What’s Missing — And What You Should Do About It
The dashboard’s biggest limitation is the absence of click data. You can see that your page was cited, but not whether anyone clicked through to it. This is a critical gap because AI citations function fundamentally differently from traditional blue links. A citation in an AI answer might satisfy the user’s need entirely within the response, generating zero clicks. Or it might signal authority and drive disproportionate downstream engagement.
Without click-through metrics, practitioners will need to triangulate. Here’s a practical framework for doing that:
The AI Citation Triangulation Method: Cross-reference your Bing AI citation data with your own server-side analytics. Filter for referral traffic from Bing Copilot and Edge Sidebar. Overlay that against cited pages in the dashboard. Where you see citations but no referral traffic, the AI answer is likely satisfying user intent without requiring a visit. Where you see both, you’ve found your highest-value AI-visible pages.
Also consider using UTM parameters and referer analysis to separate Copilot-driven visits from traditional Bing organic traffic. The distinction matters because the user behavior patterns — and conversion potential — differ.
Hidden HTTP Homepage Bug: The Invisible Page Breaking Your Site Name
Google’s John Mueller shared a troubleshooting case on Bluesky that reads like a ghost story for technical SEOs: a leftover HTTP homepage — one the publisher never saw — was corrupting the site name and favicon displayed in search results.
The mechanics are straightforward but devious. The site was fully HTTPS. The publisher saw only the HTTPS version in their browser. But the server still had an accessible HTTP homepage — typically a default server page — that contained different content. Because Chrome silently upgrades HTTP requests to HTTPS, the publisher had no way to encounter the ghost page through normal browsing.
Why Googlebot Sees What You Don’t
Here’s the technical detail that makes this dangerous: Googlebot does not perform Chrome’s automatic HTTP-to-HTTPS upgrade. When Googlebot crawls the HTTP version of your domain, it gets whatever is actually served at that URL. If that’s a server-default Apache or Nginx welcome page rather than your real homepage, Google may use that content to determine your site name and favicon.
Mueller’s advice was to use curl from the command line to see the raw HTTP response — bypassing Chrome’s auto-upgrade behavior. You can also run a Live Test through the URL Inspection tool in Search Console to see exactly what Googlebot retrieved and rendered.
What Most People Get Wrong About This Issue
The original coverage frames this as a rare edge case. I’d push back on that characterization. In my experience, leftover HTTP configurations are far more common than most site owners realize, especially in these scenarios:
Server migrations: When moving between hosting providers, the new server’s default HTTP configuration may not match your old one. If you didn’t explicitly configure HTTP-to-HTTPS redirects at the server level (not just in your CMS), the default page may be exposed.
CDN and reverse proxy setups: Cloudflare, AWS CloudFront, and similar services handle SSL termination at the edge. But the origin server may still respond to HTTP requests with a default page, and Googlebot sometimes fetches from origin.
Multi-domain setups: If your server hosts multiple domains, a misconfigured virtual host can serve another site’s homepage — or a default page — on the HTTP version of your domain.
Containerized deployments: Docker and Kubernetes setups may expose a default HTTP response on port 80 that has nothing to do with your actual website.
A Quick Diagnostic Checklist
Rather than waiting for this bug to surface in your search results, run a proactive check. Open a terminal and execute: curl -I http://yourdomain.com (note: HTTP, not HTTPS). If the response doesn’t return a 301 redirect to your HTTPS URL, you have a potential problem. Then run curl http://yourdomain.com to see the full response body. If it’s anything other than a redirect or your actual homepage, fix it immediately by configuring a server-level 301 redirect from HTTP to HTTPS for all paths.
Google’s documentation on site names specifically addresses duplicate homepages — including HTTP/HTTPS variants — and recommends using identical structured data on both. But the better solution is to eliminate the HTTP version entirely through proper redirects.
Googlebot’s 2 MB Crawl Limit: Why It’s (Probably) Not Your Problem
Google recently updated its Googlebot documentation to clarify that the crawl limit for supported file types is 2 MB, while PDFs get a 64 MB limit. This prompted concern in the technical SEO community, partly because older sources referenced a 15 MB figure.
New analysis of HTTP Archive data, covered by Search Engine Journal’s Roger Montti, puts those concerns into perspective. The overwhelming majority of real-world webpages fall far below the 2 MB threshold.
Mueller himself shared Web Almanac data on Reddit showing that the median mobile page has about 33 KB of HTML, and 90% of pages come in under 151 KB. That’s roughly 7.5% of the limit.
The Nuance the Headlines Miss
Here’s where the conversation needs more precision. The 2 MB limit applies to the initial HTML fetch — the raw document Googlebot downloads before rendering. It does not apply to the total weight of a fully rendered page (which would include images, CSS, JavaScript, fonts, and other resources). This distinction matters because many SEOs confuse “page size” with “HTML document size.”
A page that weighs 8 MB in a browser (after loading all assets) might have an HTML document of only 50 KB. That page has zero crawl limit concerns. Conversely, a page with massive inline SVGs, embedded JSON-LD data, or inline JavaScript could have a relatively lightweight total page weight but an oversized HTML document.
Who Actually Needs to Worry
For 99.99% of websites, this limit is irrelevant — a point made by Dave Smart of Tame the Bots, who also updated his fetch-and-render tool to simulate the cutoff. But certain edge cases deserve attention:
Single-page applications (SPAs) with server-side rendering that inline large data payloads into the initial HTML. Think e-commerce category pages that embed hundreds of product objects in JSON within the document.
Pages with massive inline CSS or JavaScript. If your build process doesn’t externalize these resources, they inflate your HTML document size.
Auto-generated pages with repetitive markup patterns — comparison tables, directory listings, or data-heavy dashboards.
CMS plugins that inject large blocks of configuration data, schema markup, or tracking scripts directly into the HTML.
If you’re concerned, Smart’s Tame the Bots fetch-and-render tool now simulates the 2 MB cap. Mueller endorsed the tool on Bluesky as a practical way to check.
A Broader Trend Worth Tracking
The reduction from a commonly cited 15 MB to a documented 2 MB is significant in principle, even if it’s irrelevant in practice. It suggests Google is optimizing crawl infrastructure for efficiency. In an era where Googlebot is also feeding content to AI systems — not just the traditional index — tighter resource constraints make engineering sense. Expect more documentation clarifications like this as Google continues to make its crawling systems more transparent.
The Real Theme: Diagnostic Visibility Is the New SEO Battleground
Step back from the individual stories and a broader pattern emerges. The most consequential shift in SEO right now isn’t a new algorithm update or ranking factor. It’s the expansion — and uneven distribution — of diagnostic visibility.
Bing gives you AI citation data that Google doesn’t. A curl command reveals what Chrome hides. HTTP Archive data answers what documentation couldn’t. In each case, the practitioners who know where to look, and how to synthesize imperfect data across platforms, gain an advantage that no amount of keyword optimization can match.
This is the emerging skill gap in SEO. The discipline is less about knowing which lever to pull and more about knowing which instrument to read. The professionals who build systematic diagnostic workflows — who check HTTP as well as HTTPS, who cross-reference Bing AI citations with their own analytics, who validate documentation claims against real-world data — will consistently outperform those who rely on surface-level tools alone.
FAQ
What are grounding queries in Bing Webmaster Tools?
Grounding queries are the specific phrases that Bing’s AI used when retrieving your content to construct Copilot answers. They function as the AI-era equivalent of traditional search queries, showing what the model was looking for rather than what a user typed. You can find them in the new AI Performance dashboard in Bing Webmaster Tools.
How do I check if a hidden HTTP homepage is affecting my site in Google?
Run curl -I http://yourdomain.com in your terminal. If the response isn’t a 301 redirect to your HTTPS URL, you may have an exposed HTTP homepage. Follow up with curl http://yourdomain.com to see the full page content. If it shows anything other than your real homepage or a redirect, configure a server-level 301 redirect from HTTP to HTTPS.
What is Googlebot’s HTML crawl limit?
Googlebot’s crawl limit for standard supported file types, including HTML, is 2 MB. PDFs have a separate 64 MB limit. The limit applies to the initial HTML document only — not to the total rendered page size, which includes images, CSS, and JavaScript loaded separately.
Does the 2 MB Googlebot limit affect most websites?
No. According to Web Almanac data shared by Google’s John Mueller, 90% of pages have less than 151 KB of HTML. The median mobile page is approximately 33 KB. Only pages with extremely bloated inline markup, embedded data, or un-externalized scripts would approach the threshold.
Should I care about Bing AI citations if my Bing traffic is low?
Yes. Copilot reaches users across Windows, Edge, and Microsoft 365 — far beyond Bing.com search. Grounding query patterns in Bing may also reflect how other AI systems retrieve and cite content. Additionally, the dashboard provides a competitive intelligence signal about your AI visibility that Google Search Console doesn’t currently offer.
How does Google’s AI Overviews reporting differ from Bing’s AI citation dashboard?
Google includes AI Overviews and AI Mode data within Search Console’s existing Performance reports but does not break it out as a separate, citation-specific dashboard. AI Overviews also assign all linked pages to a single position, which limits page-level performance insights. Bing’s dashboard provides dedicated, per-page citation counts and grounding query data as a distinct reporting surface.
