The fifth rerun buys almost nothing

Measurement design in AI visibility is the decision about how to spend a fixed query budget across four separate things: repeats of one prompt, paraphrases of that prompt, models, and languages. A new variance decomposition finds that most teams put the budget into the one that returns the least.

#What the standard setup looks like

Current practitioner guidance is fairly consistent. SE Ranking's guide to choosing prompts to track recommends starting with 20 to 40 prompts spread across journey stages, quotes Kevin Indig recommending roughly five consecutive runs of the prompt set once a week, and advises tracking for at least 30 days before drawing conclusions.

That shape assumes the noise lives inside the prompt. Ask the same question five times, average, and the wobble cancels out. It is a reasonable assumption. It is also, according to the first study to formally partition the variance, the wrong place to spend most of the money.

#Four sources of movement, not one

Where Does the Noise Come From? A Variance-Components Decomposition of Non-Determinism in LLM Brand Answers applies generalizability theory, a measurement framework built for exactly this problem: splitting a score's total variance into named facets, then forecasting how reliability changes if you buy more of each one.

The corpus is fully crossed: 12,933 responses covering 20 Central and Eastern European brands, 8 languages, and 3 models (GPT-5.2 and Gemini 3 Flash in parametric mode, Perplexity in grounded retrieval), with a stability subset of 1,435 cells resampled about five times each. The measured outcome is per-response sentiment polarity toward the brand.

Under the paper's headline decomposition, query language accounts for 26.5% of the variance of a single response, while brand identity accounts for 1.5% (ICC 0.0146). Pure resampling is 34.8%, the brand-in-context interaction 29.6%, and brand by language 8.6%. Brand by model and brand by prompt sit near zero.

Read the first two numbers together. The wording and language of the question moves a single AI answer roughly seventeen times as much as which brand is being asked about. One answer is close to useless as evidence about a brand.

#Why the biggest component is not the best buy

Resampling is the largest single slice, so the intuitive move is to add repeats. That inverts the logic. Resampling noise is unsystematic, which is precisely why it collapses quickly under averaging and why the fifth repeat has already taken most of what is available. Language and model are systematic offsets. They do not average away no matter how many times you rerun the same prompt in the same language on the same model, because that design never samples them.

The paper's decision study prices this directly. Starting from a base design of one language, one model, five paraphrases and five repeats (25 queries), each expansion buys the following reduction in relative-error variance, per the D-study allocation:

Expansion	Extra queries	Variance reduction
5 more repeats	25	0.00030
5 more paraphrases	25	0.00125
2 more models	50	0.00167
3 more languages	75	0.00462

The authors' summary of that table is that language diversity reduces relative-error variance about fifteen times as much as five more repeats. Their allocation rule ranks the facets languages, then models, then paraphrases, then repeats, and states that the least-cost design maximizes language and model coverage before adding a sixth repeat.

#The ceiling is lower than the dashboard suggests

The same study reports generalizability coefficients for whole designs. Figures below are from the reliability ladder in that paper.

Design	Queries	Ranking reliability
Single answer	1	0.010
20 repeats of one prompt	20	0.020
8 languages, 1 model, 1 prompt, 10 repeats	80	0.132
8 languages, 3 models, 15 paraphrases, 1 repeat	360	0.347
Full crossed design	1,800	0.362

Twenty repeats of one prompt land at 0.020. Spreading a smaller budget across languages, models and paraphrases with a single run each lands at 0.347. Going from that 360-query design up to the full 1,800-query one, where the extra volume is almost entirely reruns, adds about 0.015.

The other thing that table says is that even the best design tops out near 0.36 for ranking brands, with the absolute-score coefficient around 0.10. That is well below any conventional threshold for a reliable instrument. Brand rankings out of these panels are directional evidence, not scores you should put a decimal place on.

#Paraphrases and questions are different axes

The paraphrase facet above varies the wording of one question. Covering the question space is separate work, and there is now data confirming the two are not interchangeable.

PersonaGen-1M, a demand-side buyer corpus of over one million synthetic personas carrying 5,160,046 search queries, tested whether a persona's query set behaves like restatements of a single need. It does not. Mean pairwise lexical overlap between two queries from the same persona was 0.042, against 0.041 for queries drawn from different personas. A buyer's questions are distinct needs, not variations on one.

That corpus splits intent as 78.3% informational, 17.4% commercial, and 4.3% transactional, with commercial intent running higher in B2B contexts than B2C. A 30-persona FinTech pilot found 97.3% of answers named at least one brand, averaging 5.01 brands per answer across 530 distinct brands. Shortlists are crowded, so small differences in your prompt set change which competitors you appear next to.

#The caveats that matter

The outcome variable in the decomposition is sentiment polarity, not citation share or mention rate. The allocation method transfers cleanly; the exact percentages should not be assumed to. The paper also lists real limits: 91.9% of sentiment scores were exactly neutral, so the Gaussian model is an approximation, confidence intervals are deferred to a later version, decoding ran at temperature 0.3 (making the 34.8% resampling share a lower bound), and the corpus is 20 brands in one collection window. Both papers cited here share an author, so this is one research programme rather than independent replication.

#What to change on Monday

Stop at five repeats. Past that, extra runs are the most expensive reliability you can buy.

Spend the freed budget on facets that do not average away. If you sell in multiple languages, language coverage is the single largest lever and most panels ignore it. If you sell in one, that lever is off the table, so buy model coverage and genuinely different buyer questions instead.

Report per engine rather than as one blended score, because model identity is a systematic facet and blending hides it.

Ask your vendor which facets they cross and how many runs sit behind each number. A tool that runs five repeats of 30 prompts on one model in one language is sampling one corner of the design, however many data points the dashboard shows.