Most AI crawlers don't run your JavaScript

JavaScript rendering is the step where a browser runs a page's scripts to build the content a human finally sees, and most AI crawlers skip that step entirely. When a bot like GPTBot, ClaudeBot, or PerplexityBot requests a URL, it reads the raw HTML the server returns and stops. Whatever your JavaScript would have injected after load is simply not in the copy the crawler keeps.

This matters because an AI answer can only quote text the engine actually fetched. If your headline, product description, pricing, or FAQ only appears after a client-side framework hydrates the page, the crawler sees an empty shell where that content should be. The page looks complete in your browser and blank to the model.

#What "fetch but not render" means

There are two distinct jobs a crawler can do. Fetching is the cheap part: make an HTTP request, get bytes back. Rendering is the expensive part: spin up a headless browser, execute the scripts, wait for network calls to resolve, and read the final DOM. Search engines built rendering pipelines over years because so much of the web moved to client-side rendering. Most AI crawlers have not.

Vercel analyzed traffic from the major AI bots and found that none of them currently render JavaScript, a group that includes OpenAI's GPTBot and OAI-SearchBot and Anthropic's ClaudeBot (Vercel, "The rise of the AI crawler"). These bots do still download script files as text. In Vercel's data, JavaScript made up 11.50% of ChatGPT's fetches and 23.84% of Claude's, but downloading a .js file is not the same as running it. They store the file; they do not execute it to build the page.

Look at where the content actually is. ChatGPT's crawler spent 57.70% of its fetches on HTML, and Claude's spent 35.17% on images. The HTML is the document they treat as the page. If your meaningful text is not in that HTML response, it is not in the version the engine reasons over.

#The one real exception

Google's family of crawlers is the outlier. Gemini draws on the same rendering infrastructure as Google Search. Google's Martin Splitt put it plainly: "Gemini uses the same rendering services as Googlebot". That means content Googlebot can render, Gemini can generally see, with the usual caveats around timing, blocked resources, and queue delays that have always applied to Googlebot.

So the practical map looks like this. Google's surfaces can read client-rendered pages. Most everything else, including the crawlers feeding ChatGPT, Claude, and Perplexity, reads only what is in the initial HTML. Optimizing for one does not cover the other.

#What survives, what disappears

The dividing line is when the content enters the HTML.

Content that survives a no-render crawl:

• Text, headings, and links present in the server's initial HTML response.
• Server-side rendered (SSR) or statically generated (SSG) pages, where the framework ships finished markup.
• Structured data and metadata embedded in the source HTML, including JSON-LD in the document.
• Content already serialized into the page by the server before it reaches the client.

Content that disappears:

• Anything injected only after a client-side framework mounts, the classic single-page-app pattern where the initial HTML is little more than an empty <div id="root">.
• Text loaded by a fetch call that fires on the client after page load.
• Tabs, accordions, and "load more" sections whose content is requested only on interaction.
• Anything gated behind a click, scroll, or other event a non-rendering bot will never trigger.

Google's own guidance describes search rendering as a separate, deferred stage from crawling, which is exactly why client-side content can lag or fail to appear even on the engine that does render (Google Search Central, "JavaScript SEO basics"). For crawlers that never render at all, there is no deferred stage to catch up. The first fetch is the only fetch.

#How to check your own pages

You do not need special tooling to see what a non-rendering crawler sees. Request the raw HTML and read it as text:

curl -s https://example.com/your-page > raw.html

Open raw.html and search for a sentence you expect to find: your main heading, a key product claim, a price. If the words are there, an AI crawler can see them. If the file is mostly script tags and an empty container, your content lives only in the rendered DOM, and most AI crawlers are not rendering it.

In our own audits at Crawlmind, the gap shows up as a difference between the static fetch and the rendered fetch of the same URL. When the rendered version has paragraphs of text the raw HTML does not, that delta is precisely the content most AI engines will miss.

#What to do about it

The fix is to move important content into the initial HTML response rather than relying on the client to build it.

• Render on the server. SSR or static generation puts finished markup in the first response. Most modern frameworks support this directly; the question is whether your routes actually use it for the pages that matter.
• Prioritize by page value. You do not have to convert an entire app. Identify the URLs you want cited (product pages, docs, comparison pages) and make sure their core text ships in the HTML.
• Keep critical content out of interaction-gated regions. If an answer lives inside a tab or an accordion that loads on click, a non-rendering crawler will not open it. Put the substance in the document.
• Verify after framework or infra changes. A migration, a new rendering mode, or an edge configuration can quietly flip a page from server-rendered to client-rendered. Re-run the curl check on your highest-value pages when the stack changes.

#The summary

Most AI crawlers fetch raw HTML and do not execute JavaScript, so any content your scripts build on the client is invisible to them. Google's Gemini is the meaningful exception because it shares Googlebot's rendering. For everyone else, the rule is simple: if the text is not in the HTML the server returns, it is not in the answer. Check your key pages with one curl command, and make sure the content you want quoted is in the document before the first script runs.