ClaudeBot: how to allow Anthropic to crawl your site

ClaudeBot is Anthropic's primary web crawler. It fetches publicly available pages to gather data used for training future Claude models and to retrieve content when a Claude user explicitly browses a URL. Allow ClaudeBot in robots.txt if you want your content to be eligible for inclusion in Anthropic's training corpus and for citation in Claude.ai answers; block it if you don't.