AI visibility is a distribution, not a number

AI visibility is a distribution, not a fixed value: the same prompt sent to the same engine twice will name different brands and cite different pages, so any single check is one draw from a range rather than a reading of where you stand.

That sounds like a technicality until you see how wide the range is.

#The overlap is roughly a third

A study from the University of St. Gallen, Don't Measure Once: Measuring Visibility in AI Search (GEO), tracked 8 prompts across four verticals on ChatGPT, Gemini, Google AI Mode, and Perplexity across a 45-day collection window in early 2026. Day-to-day overlap in the set of cited sources, measured with Jaccard similarity, averaged between 34% and 42% depending on the vertical. Roughly two thirds of the sources an engine cites today are gone tomorrow.

Brand mentions held up better but still moved. Brand-set overlap ran 45% to 59% day over day. Rank-biased overlap, which weights the top of a list more heavily than the tail, came in lower than Jaccard in every vertical, so the order in which brands and sources appear shifts as well as the membership.

#The churn comes from the model, not the index

The obvious explanation is that the web changed underneath the engine: new pages, fresh crawls, an algorithm update. The paper tests that directly by re-running the same prompts on the same day, up to 10 times per engine and prompt, then comparing only pairs recorded within 24 hours of each other. Across 3,409 pairwise source comparisons, same-day overlap averaged 32% to 43%. That is essentially the same range as the day-to-day figure.

This is the load-bearing result. If waiting a day and waiting five minutes produce the same amount of churn, then most of the movement on an AI visibility dashboard is not the market shifting. It is sampling noise.

The mechanism is documented well outside the search context. In Non-Determinism of "Deterministic" LLM Settings, researchers ran five language models configured for deterministic output across eight tasks, ten runs each, and recorded accuracy swings of up to 15% between runs with a best-case to worst-case gap as wide as 70%. Identical settings do not produce identical output, largely because inference batching and floating-point reduction order are not held fixed between requests.

Retrieval adds a second layer on top of that. Google's own documentation on AI features and your website states that AI Mode and AI Overviews may use different models and techniques, so the set of responses and links they show will vary, and that both may use a query fan-out technique that issues multiple related searches to build one answer. Different fan-out means a different evidence pool, which means different citations.

Ahrefs measured how far apart two Google surfaces land on the same question. Across 730,000 response pairs, AI Overviews and AI Mode reached about 86% semantic similarity while sharing only 13.7% of their citations. The conclusion held. The sources credited for reaching it did not.

#Engines are not equally noisy

Same-day repeatability varies a lot by engine, which is why a single blended "AI visibility score" hides the thing you need to know.

Engine	Source overlap (Jaccard)	Brand overlap (Jaccard)
ChatGPT	0.23	0.44
Perplexity	0.28	0.49
Gemini	0.51	0.41
Google AI Mode	0.32	0.38

Mean pairwise similarity within a 24-hour window, from the St. Gallen dataset.

Gemini repeated its source set about twice as consistently as ChatGPT. ChatGPT also carried a separate problem: web search never fired on 57.8% of its runs, and a prompt that never triggers retrieval cannot cite you regardless of how good the page is.

Concentration differs too. The study reports a mean citation Gini coefficient of 0.715 across verticals and engines, with Google AI Mode the most concentrated at 0.782 and Perplexity the most even at 0.671. A handful of domains takes most of the visibility, and how steep that curve is depends on which engine you are looking at.

#What a defensible measurement takes

The authors run a bootstrap convergence analysis to answer the practical question of how much sampling is enough. Three numbers come out of it.

Runs per prompt per day. Standard error on a per-brand detection rate falls below 0.10 at seven runs and below 0.08 at eight. Source-level coverage converges more slowly and needs eight. One query a day is not a measurement.

Prompts per topic. Per-prompt overlap in the study ranged from under 0.2 to above 0.8. A tracker built on one or two prompts measures those prompts, not your topic. Specific product questions were answered more consistently than broad, generic ones, so a prompt portfolio needs both kinds or it will look artificially stable.

Window length. With roughly 65% of cited sources turning over from one day to the next, a week is too short to separate signal from noise. The rolling-window analysis puts standard error below 0.10 at 10 days and below 0.05 at 24 days. Two to four weeks of rolling aggregation is the floor for a per-brand figure you would show an executive.

#What to change in reporting

Make brand presence the headline KPI, not cited URLs. Brand-level stability beat source-level stability in both the daily and the same-day analysis, so brand presence is the more honest number. Keep source tracking underneath it to learn which pages earn inclusion, but a per-URL citation count does not belong on the dashboard as a trend line.

Attach an interval to every figure. A citation share quoted with no sample size and no error bar is not reportable. The same figure quoted as an average over seven runs a day across three weeks, with a stated confidence interval, is.

Set baselines per engine. A decline on Google AI Mode and a decline on Perplexity do not carry the same weight when their underlying variance is this different. One alert threshold across all four engines will fire constantly on the noisy ones and stay silent through real losses on the stable ones.

Treat a single-run check as a hypothesis. It tells you what one answer looked like once. That is genuinely useful for reading how an engine frames your category, or for catching a page that is obviously broken or missing. It is not evidence that you gained or lost ground.

#The part that does not change

None of this makes optimization pointless. The original GEO paper from Princeton and IIT Delhi showed in a controlled setup that content changes do move visibility, with its strongest methods producing a relative improvement of 30% to 40% on a position-adjusted visibility metric. The effect is real. It is just smaller than the noise floor of a one-shot check, which is why teams keep concluding that GEO does nothing, or that it works spectacularly, on the strength of a single Tuesday afternoon spot check.

Sample it like the probabilistic system it is, and the signal shows up.