AI engines cite a different web
Crawlmind Engineering··5 min read
The AI citation pool is the set of domains answer engines actually link to when they cite sources, and it is not the same pool that sits behind a traditional results page. A large measurement study puts the overlap at well under half, which means auditing your classic rankings tells you less about your AI visibility than most teams assume.
The study is Source Coverage and Citation Bias in LLM-based vs. Traditional Search Engines, posted to arXiv in December 2025. It analyzed 55,936 queries across six LLM search engines and two traditional ones, collected daily from July 16 to August 10, 2025 from US vantage points. The LLM side covered ChatGPT, Gemini, Perplexity, Grok, Google AI Mode and Copilot Search. The traditional side was Google and Bing.
#Two pools, one third shared
The headline finding is about set membership, not ranking. Across the whole corpus, only 38% of domains appear in both, whereas 37% are unique to LLM-SE results. The remaining quarter shows up only in traditional search.
Read that as a competitive map. Roughly a third of the domains earning citations in AI answers would never have appeared in a classic SERP audit for the same queries. Another quarter of the sites that rank are invisible on the AI side. If your competitive set was assembled from Google results, you are looking at a sample that misses about a third of who is actually being cited.
This is a different claim from the cross-engine overlap numbers that circulate in GEO writeups. Those compare AI engines to each other. This one compares the AI layer as a whole against the search layer as a whole, and it says the two are drawing from substantially different inventories.
#Fewer slots per answer
The second structural difference is volume. LLM search engines source fewer URLs (mean 4.3) and domains (mean 3.4) compared to traditional engines (mean 10.3 URLs and 7.3 domains).
So the pool is wider but each answer is narrower. A results page gives you roughly seven distinct domains to be one of. An AI answer gives you three or four. That compression is the reason citation share behaves so differently from rank share: small changes in selection move a much larger fraction of the visible surface.
Some engines cite far less than that average implies. The study found Grok and Gemini are the least likely to cite external sources, with 82% and 38% of their responses containing no cited websites. An engine that answers without linking is not a channel you can win by any on-page change. It is worth knowing which of your tracked engines fall in that category before you set expectations on citation counts.
#What predicted selection
The authors trained a classifier on domain and page features to see what separates cited from uncited sources. The best model was XGBoost, achieving a weighted F1 score of 0.758, with the top features by mean SHAP value being Tranco ranking (0.923), number of outlinks (0.799), .com top-level domain (0.623) and subdomain count (0.189).
Tranco is a research-oriented top sites ranking built to resist the manipulation that made older popularity lists unreliable for measurement work. Using it as the strongest single feature says domain popularity still dominates source selection, which is a deflating result for anyone hoping that answer engines reward page craft over site authority.
There is a twist in that finding, though. Popularity predicts selection, but the AI engines reach deeper into the tail than search does. Mean Tranco ranks of cited domains came in at 69,054 for Copilot and 64,121 for Gemini, against 41,428 for Google and 48,125 for Bing. A higher number means a less popular site. Popularity is the best available predictor inside the pool, and the pool itself extends further down than the first page of Google does. Both things are true, and the second one is where a mid-size site has room.
The page-level signals point the same direction as most retrieval guidance. LLM-cited pages averaged a grade level of 14.57 versus 18.24 for traditional results, and a reading ease score of 24.15 versus 12.32, with statistically significant differences in semantic tag usage and DOM nesting depth. Simpler prose, more structured markup. Neither is a trick, and neither will substitute for the domain-level features that dominated the model.
One concentration risk is worth calling out. For Gemini, the study found Wikipedia accounts for 41% of its citations. If your category has a thin or outdated Wikipedia presence, a large share of one engine's evidence layer is being written by people who are not you.
#What to do with this
Three practical changes follow.
Audit against the AI pool, not the SERP. If you build your competitor list, your gap analysis, or your content brief from classic rankings, you are working from a set that misses roughly a third of the domains being cited. Pull the sources from the answers themselves.
Separate access from eligibility. Crawler permissions are not a ranking lever, but they are a gate. OpenAI documents OAI-SearchBot as the crawler that surfaces sites in ChatGPT search, independent of the training crawler GPTBot, and states that sites opted out of OAI-SearchBot will not appear in ChatGPT search answers. Check that specific agent before you conclude a page lost on merit.
Treat outlinks as a real signal. The number of outlinks on a page was the second strongest feature in the model. Pages that cite their sources look different to these systems than pages that assert. That is cheap to fix and it matches what the retrieval layer is optimizing for.
#The caveats that matter
This is a snapshot: US vantage points, a 26-day window in mid-2025, and six engines whose retrieval stacks have all shipped changes since. The feature analysis is correlational. It identifies what distinguishes cited domains, not what causes a citation, and a popularity feature that strong will absorb a lot of variance that belongs elsewhere.
What holds up is the structural part. Two overlapping but distinct inventories, half as many domain slots per answer, and a selection process that still leans on site-level authority while reaching further down the tail than search does. That shape is unlikely to have flipped in a year, and it is enough to change how you pick what to measure.
Related field notes
September 24, 2026 · 6 min
Google pays for grounding, not for links
Google's AI contribution pilot pays when a page shapes an answer, not when it is linked afterward. That rule says a citation count measures the wrong thing.
September 23, 2026 · 5 min
A browser agent is not a crawler
Agentic browsing runs inside the user's own session, so robots.txt, bot allowlists and crawler analytics all miss it entirely.
September 22, 2026 · 4 min
An MCP endpoint is not a discovery channel
NLWeb and MCP make your site answerable by agents that already found you. Nothing on the open web is hunting for a /mcp route yet.
Share or discuss
New posts, no spam. Roughly monthly. Unsubscribe with one click.