We respect your privacy.

We use strictly necessary cookies to keep you signed in and to protect against CSRF. With your permission we also use a small amount of first-party analytics to improve the product. We do not sell your data and we do not use third-party advertising trackers. See our cookie policy and privacy policy .

Crawlmind
Home/Glossary/GPTBot

AI crawlers

GPTBot

Updated 2026-05-17

GPTBot is OpenAI's primary web crawler. It fetches publicly available pages to gather data for training future GPT models and, in some configurations, to ground answers in ChatGPT. Sites can allow or disallow it via robots.txt with a User-agent: GPTBot directive.

Allow GPTBot

User-agent: GPTBot
Allow: /

Block GPTBot

User-agent: GPTBot
Disallow: /

See also

OAI-SearchBot (ChatGPT search), ChatGPT-User (user-triggered fetches).

Related

Where this comes up

See how your site scores on GPTBot + every other AI-discoverability signal.

Free audit