AI crawlers
anthropic-ai
Updated 2026-05-17
anthropic-ai is the legacy user-agent name for Anthropic's web crawler. Anthropic has since standardised on ClaudeBot, but anthropic-ai directives in robots.txt are still respected for backwards compatibility. Allow or block both for the most defensible posture.
Related
Where this comes up
See how your site scores on anthropic-ai + every other AI-discoverability signal.
Free audit