AI crawlers
Bytespider
Updated 2026-05-17
Bytespider is ByteDance's crawler, used to feed training data and live retrieval for Doubao and TikTok Search. It's known for aggressive crawl rates, and many sites block it by default for bandwidth reasons.
Where this comes up
See how your site scores on Bytespider + every other AI-discoverability signal.
Free audit