Foundations
AI crawlers
GPTBot: how to allow OpenAI to crawl your site
What GPTBot is, what it's used for, and the exact robots.txt to allow or block it.
Read →
ClaudeBot: how to allow Anthropic to crawl your site
What ClaudeBot is, the difference between ClaudeBot, anthropic-ai, and Claude-Web, and the exact robots.txt to allow or block each.
Read →
The complete list of AI crawlers in 2026
Every AI crawler operating against the public web as of May 2026 — user-agent, operator, what it fetches, whether it honors robots.txt, and the recommended policy.
Read →
GEO tactics
Schema markup for AI search: a 2026 reference
Which schema types matter for ChatGPT, Perplexity, Claude, and Gemini — and which Google-era schemas are now noise.
Read →
AI citation readiness: write pages AI engines cite
The structural and editorial signals that determine whether AI engines cite your page in their answers.
Read →
Perplexity SEO: how to be cited by Perplexity
Perplexity is the most citation-friendly AI engine in 2026. Here's how its index works and what to optimize.
Read →