We respect your privacy.

We use strictly necessary cookies to keep you signed in and to protect against CSRF. With your permission we also use a small amount of first-party analytics to improve the product. We do not sell your data and we do not use third-party advertising trackers. See our cookie policy and privacy policy .

← All posts

A repeatable AI-visibility content audit

Crawlmind Engineering··5 min read

An AI-visibility content audit is a repeatable review that checks whether AI assistants can reach a page, read it as plain text, extract a clean answer from it, and cite it, then measures whether they actually do. It is the AI-answer version of a technical SEO audit, run on a fixed cadence against a fixed checklist so the results are comparable over time.

The reason to write it down as a checklist rather than eyeball it: the engines do not agree with each other. In an analysis of 680 million citations from August 2024 to June 2025, Wikipedia made up 47.9 percent of ChatGPT's top ten most-cited sources, while Reddit made up 46.7 percent of Perplexity's top ten and 21 percent of Google AI Overviews' top ten (Profound). A page that reads as authoritative to one system can be invisible to another. If your audit is a gut check, that divergence hides in the average. A scored checklist forces you to look at each layer separately and each engine separately.

Here is the checklist we run, grouped into five passes. Score each item, note the fix, and re-run the whole thing every 30 to 90 days.

#Pass 1: access

Before anything else, confirm the AI crawlers can fetch the page at all.

  • Check robots.txt for the agents that matter: GPTBot and OAI-SearchBot (OpenAI), ClaudeBot and Claude-SearchBot (Anthropic), PerplexityBot, Google-Extended (Gemini), and Meta-ExternalAgent. A blanket Disallow here is the most common reason a page never appears in answers.
  • Confirm the page returns a 200 and is not gated behind a login, an interstitial, or an aggressive rate limit that trips bot traffic.
  • Check for a noindex tag or an X-Robots-Tag header that a migration left behind.

#Pass 2: retrievability

An accessible page is not the same as a readable one. Most AI crawlers fetch raw HTML and stop. A Vercel and MERJ study of GPTBot found zero JavaScript execution across the fetches they analyzed (Vercel). Client-rendered content that only appears after JavaScript runs is, to those crawlers, a blank shell.

  • View the page source (not the rendered DOM) and confirm your main content, headings, and answers are present in the initial HTML.
  • If the site is a single-page app, verify server-side rendering or static generation is producing real markup for the routes you care about.
  • Confirm the important text lives in the HTML body, not baked into images or loaded from a separate client call.

#Pass 3: answerability

This is where most content fails, because it was written to rank, not to be quoted. AI systems lift sentences and short passages, so the audit checks whether each page hands them a clean unit to lift.

  • Does the page open with a direct, self-contained answer to the question it targets, in the first paragraph? A reader (or a model) should get the answer without scrolling.
  • Are claims stated as complete sentences that survive being pulled out of context? "It costs 20 percent less" fails; "Plan A costs 20 percent less than Plan B" survives.
  • Are headings phrased as the questions people actually ask, so a model can map a query to a section?
  • Do comparisons and specs live in tables or tight lists rather than buried in prose? Comparative content earns disproportionate visibility: in one study, comparative queries produced 2.4 times more brand mentions than informational content (Semrush).

#Pass 4: trust signals

Retrievable, answerable content still needs a reason to be trusted over the next source.

  • Is authorship real and attributable? A named author with a bio and a track record beats an anonymous byline.
  • Is the publish or update date visible and honest, and does dateModified in your structured data match it?
  • Does the JSON-LD schema (Organization, Article, FAQ, Product where relevant) describe what is actually on the page? A mismatch between schema and visible content is a trust penalty, not a shortcut.
  • Are the facts sourced? Outbound links to primary sources signal the same care to a model that they signal to an editor.

#Pass 5: measurement

The first four passes are inputs. This one checks the output, and it is the pass most audits skip.

  • For your priority queries, ask each engine directly (ChatGPT, Perplexity, Google AI Mode, Gemini) and record who gets cited. Do this per engine, because, as the citation data shows, a win on one is not a win on all.
  • Track citations and brand mentions separately. They are not the same event. In the Semrush study of 3,981 domain appearances across 115 prompts in 14 countries, 62 percent were "ghost citations," where the AI used a source but never named the brand in the answer (Semrush). A page can be doing the work and getting none of the credit, and only per-query tracking surfaces that.
  • Log the answer text, not just a yes or no. The exact sentence an engine quotes tells you which passage is working and which competitor phrasing is beating yours.

#Making it repeatable

The value of an audit is in the second run, not the first. Keep the checklist stable so a July score means the same thing as an April score. Store each finding in a category with a remediation and an owner, so the audit produces a work queue rather than a report that gets read once. Re-run on a fixed cadence: monthly for a small priority set, quarterly for the full site.

Two failure modes to avoid. First, do not average across engines. A single "AI visibility" score hides the exact divergence the Profound data documents, and the divergence is where the work is. Second, do not stop at access and retrievability because they are easy to automate. Those passes are necessary and insufficient. The pages that win citations clear all five, and the last two are where the differentiation lives.

Run it once to get a baseline. Run it again in a quarter to find out whether anything you changed moved the answer.

Related field notes

Share or discuss

Field notes in your inbox

New posts, no spam. Roughly monthly. Unsubscribe with one click.