We respect your privacy.

We use strictly necessary cookies to keep you signed in and to protect against CSRF. With your permission we also use a small amount of first-party analytics to improve the product. We do not sell your data and we do not use third-party advertising trackers. See our cookie policy and privacy policy .

← All posts

Count verified mentions, not mentions

Crawlmind Engineering··5 min read

A verified mention is one where your brand appears in an AI answer and the claim attached to your name is accurate, supported by the source the model cited, and pointing at a URL that actually resolves. A raw mention is your name appearing in the text, nothing more. Almost every AI visibility dashboard counts raw mentions and calls the number visibility.

The gap between the two is not a rounding error, and a paper published in June 2026 gives it a size.

#What the study measured

The Per-Entity Bias Mapping paper (arXiv 2606.21595) probed 100 Hungarian B2B companies, split evenly into a high-salience tier and a low-salience tier, using seven query templates against two models. That produced 1,400 probe runs and 2,062 verifiable source references, collected in a single session on May 22, 2026 (arXiv 2606.21595).

The important scoping decision: retrieval-augmented systems like Perplexity were excluded so the authors could isolate what the models produce from parametric memory alone (arXiv 2606.21595). This is the answer you get when the assistant does not go and fetch anything, which is still a large share of brand questions people ask.

Each extracted source was then checked. A source counted as fabricated when the URL did not exist or could not be resolved.

#Familiar brands get more fabrication, not less

The headline result runs against intuition. High-salience brands drew 52.69% fabricated citations, low-salience ones 37.87%, a gap of 14.82 percentage points at p=1.67e-11 (arXiv 2606.21595).

The mechanism the authors propose is straightforward once you state it. A model that knows a lot about a brand has a dense, confident region to generate from, including plausible-looking press-release URLs, filing paths, and press-office subdirectories that were never real. A model that knows almost nothing about an entity has less material to interpolate from, and one of the two models tested mostly declined to cite anything at all rather than invent.

That platform split matters for anyone comparing engines. Claude returned 1.34 sources per run with 48.17% fabricated; GPT-4o returned 0.13 sources per run with 22.10% fabricated (arXiv 2606.21595). GPT-4o did not have better sourcing hygiene in any useful sense. It refused more. A raw citation count would have scored Claude ten times more "visible" for the same brands.

#Your prompt set moves the number

Fabrication was not a constant property of the entity. It moved with how the question was framed. Regulatory compliance queries produced 56.77% fabrication against a factual baseline of 37.59%, with GDPR-framed and AI-ethics-framed prompts landing near 53% (arXiv 2606.21595).

If your monitoring prompt set leans toward compliance, certification, or "is X regulated" phrasing, you are sampling the highest-error region of the space. If it leans toward plain factual questions, you are sampling the cleanest. Two teams tracking the same brand with different prompt sets will disagree, for the same reason AI citation studies disagree with each other: the denominator was chosen before the measurement started.

#Grounded answers are a different population

Do not carry those fabrication rates into a conversation about live, retrieval-grounded AI search. It is the wrong population.

SE Ranking checked 145,463 URLs cited by ChatGPT and found 1.22% returned a 404, against 0.56% for Google's AI Overviews and 0.65% for the organic SERP (SE Ranking). Ahrefs looked at 16 million URLs across six assistants and put ChatGPT at 2.38% of mentioned URLs leading to error pages versus 0.84% for Google (Search Engine Journal).

Those are small percentages. They are also measuring something else: URLs an engine surfaced after retrieval, not URLs a model produced from memory. The relevant accuracy problem in grounded search is attribution rather than existence. The Tow Center tested eight AI search products over 1,600 queries and found they returned incorrect answers to source-identification questions more than 60% of the time, with Grok 3 producing broken links in 154 of 200 citations tested (Columbia Journalism Review).

Three studies, three failure rates, one shared conclusion: the presence of a citation tells you nothing about whether it holds.

#Fabricated URLs point somewhere

Of 946 fabricated sources in the Hungarian study, 478 were what the authors call a plausible mirage: the domain was reachable and returned a response, but the path did not exist (arXiv 2606.21595). Another 298 failed to resolve at all and 155 hit an access barrier (arXiv 2606.21595).

The plausible mirage category is the one you can act on, because half the time the domain being invented against is a real one. In this dataset, 248 of the 478 404-returning fabrications targeted Hungarian regulatory and government domains (arXiv 2606.21595). For a brand, the equivalent is a model inventing yoursite.com/press/2025-certification and a user clicking it.

That is measurable from your own server logs. Filter 404s by AI-assistant referrer and look for repeated hits on paths that never existed. A recurring invented path is a signal about what the model believes you publish, and a 200-status page at that path is a cheap way to convert a hallucination into a correct answer.

#What to instrument

Replace the single mention count with three columns per appearance:

  1. Claim accuracy. Is what the answer says about you true?
  2. Source support. Does the cited page actually contain the claim, or was a real URL attached to an unrelated sentence?
  3. URL status. Does it resolve to a 200?

Only rows passing all three are verified mentions. Track that as the numerator and raw mentions as the denominator, and you have a citation fidelity rate that moves for reasons you can diagnose.

On the supply side, the same paper found entity infrastructure independently protective: English Wikipedia presence reduced the odds of fabrication by roughly 63%, with an odds ratio of 0.37 over 2,062 sources (arXiv 2606.21595). The presence gap between tiers was stark, 48% of high-salience entities had some knowledge-graph presence against 12% of low-salience ones (arXiv 2606.21595). Wikidata entries, consistent naming, and a canonical facts page are not glamorous work, but they narrow the space a model has to guess in. That is the same lever behind entity clarity.

#Where this stops

The study covers 100 entities in one country, two models, and one day of testing, and its authors say so, listing single-point-in-time measurement and unaddressed name-variation effects among the limits (arXiv 2606.21595). Treat the levels as local. Treat the mechanism as portable: salience buys you generation confidence before it buys you accuracy, and monitoring that counts appearances cannot tell the difference.

A separate ICML 2026 position paper argues the governance version of this, that answer-level measurement and disclosure need to catch up to how much influence is now being bought at the answer layer (arXiv 2606.12439). The reporting version is smaller and available now. Verify the mention before you count it.

Related field notes

Share or discuss

Field notes in your inbox

New posts, no spam. Roughly monthly. Unsubscribe with one click.