Getting retrieved beats being retrieved first

Your page's position inside an AI engine's retrieved context is a much weaker lever on whether it gets cited than whether it entered the candidate set cleanly, and the most recent reproduction work suggests the "be first or be forgotten" strand of GEO advice rests on a result that does not replicate the way people quote it.

The folk theory is easy to trace. In 2023, Nelson Liu and colleagues published Lost in the Middle: How Language Models Use Long Contexts in TACL. They found that performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts." The U-shaped curve escaped the lab quickly. It became the justification for a whole category of advice: get retrieved early, because a page buried in the middle of the retrieved set is a page the model will skim past.

That inference was always carrying more weight than the paper supported. It took a finding about deliberate placement of a known-good document in a controlled QA setup and applied it to a production pipeline where nobody controls placement, retrieval is imperfect, and the model is choosing between sources rather than locating one.

#The reproduction

Lost in the Evidence? Reproducing Document Position and Context Size Effects in RAG, by Jorge Gabín, Anxo Perez and Javier Parapar, was accepted at SIGIR 2026. It is a systematic reproducibility study rather than a new-method paper, which is what makes it useful here. The authors rebuild the position experiments under a controlled framework, on contemporary models including LLaMA-3.1 at 8B and 70B, Mistral-Nemo 12B and Gemma-3.

When they re-run the lost-in-the-middle setup, the curve does not come back. Their reported result is that performance is "again nearly flat across placements", with only a slight upward trend rather than the degradation-in-the-middle pattern. The same flatness shows up in their reproduction of the follow-up "lost but not only in the middle" work.

The authors' own summary of the study is the line worth keeping: conclusions drawn from idealised setups "do not always transfer to real-world RAG pipelines".

That is not the same as saying order never matters. In the realistic half of the paper, where relevance is mediated by a retriever rather than by oracle access, ordering does interact with context size and retrieval quality. On HotpotQA, reverse ordering (worst retrieval score first) increasingly outperforms the other strategies as the context grows. The effect lives in the messy setting, not the clean one, and it points the opposite way from the advice built on the original paper.

#The variable that actually moved

If position is close to flat, something else is absorbing the variance. A separate paper puts the two effects side by side in the same experiment.

Dynamic Context Selection for Retrieval-Augmented Generation, from Maya Iratni, Mohand Boughanem and Taoufiq Dkaki at IRIT, tested five retrieved passages with the relevant one moved through the beginning, middle and end slots. Exact match came out at 0.5447, 0.5391 and 0.5567 respectively. End over middle is roughly a 1.76% improvement in exact match and 2.35% in F1, which is real but small.

In the same paper, adding a single distractor passage to a two-hop question dropped performance by more than 26%. For three-hop and four-hop questions, one distractor cost 13.5% and 14.4%.

One irrelevant neighbour in a five-document context did an order of magnitude more damage than moving the correct document from the best slot to the worst one. For anyone optimizing a page, that reframes the job. You are not competing for a seat near the front of the bus. You are competing to be in a context that has not been polluted by near-miss pages, several of which may be your own.

This lines up with something the SIGIR reproduction found on the retrieval side: with a stronger first stage, gains concentrated at small context sizes and the gaps between ordering strategies narrowed as more documents were added. Better retrieval reduces how much any of the downstream ordering questions matter.

#Where order sensitivity is real, and why you cannot see it

There is a stage where order genuinely misbehaves, and it sits above the generator. It is the listwise reranker.

Position Bias Undermines Preference Consistency in Listwise LLM-Based Reranking, by Ethan Bito, Yongli Ren and Estrid He, was accepted at RecSys 2026. The premise is that a reranker handed an unordered candidate set should not care which arbitrary order it was serialized in. The authors test that by sampling 20 permutations per query at candidate list lengths of 15, 25 and 50, then measuring pairwise preference instability, global preference inconsistency and listwise output consistency.

The result that matters for anyone reading vendor metrics: the best-scoring method was also the least stable. On MovieLens with Llama-3B, the STELLALW approach posted the highest hit rate at 5 of any method tested, and simultaneously the worst consistency scores in the table, with pairwise instability at 0.8331 and listwise output consistency at 0.1181. A plain zero-shot baseline scored a much lower hit rate but far steadier preferences.

Relevance improved. Reproducibility collapsed. If you only look at the ranking metric, the two are indistinguishable, and that is exactly the position a GEO practitioner is in when reading a citation-share number.

#The measurement lesson

The SIGIR paper's first contribution is not about position at all. It is about sample size.

The authors show that topic sampling is a major source of variance, and that small topic sets can mask or exaggerate ordering effects. At 500 topics, the spread between ordering strategies was wide enough that the apparent winner changed from sample to sample. Their calibration procedure landed on 1,000 topics for HotpotQA and 2,000 for Natural Questions before trends held steady.

They then use that to re-examine an industry study that evaluated on 1,500 open-domain questions, 500 from each of three datasets, and judged correctness with an LLM rather than string matching. Under matched settings, their reproduction found measurable sensitivity where the industry paper had reported stability.

Substitute "prompts" for "topics" and this is the AI-visibility dashboard problem exactly. A panel of a few hundred prompts, scored by an LLM judge, run once, is enough to produce a confident ordering of your competitors that will reorder itself next week. We have made this point before about measurement variance in AI visibility, and the SIGIR work supplies a calibration method rather than a warning.

#What to do with this

Stop treating retrieval order as an optimization target. You do not control it, the evidence that it decides answers is weaker than the citations to it suggest, and the one paper that measures ordering and distractors together says the neighbours matter roughly ten times more than the slot.

Three things follow. First, spend the effort on entering the candidate set for the sub-queries that matter, because a page that is never retrieved has no position to optimize. Second, prune your own near-duplicates, because on a query where three of your pages half-answer the question, you are supplying the distractors. Third, when a tool reports that a change moved your citation share, ask how many prompts, how many runs, and judged by what. A number that flips on resampling is not a result.