Bytespider (ByteDance)

ByteDance's crawler, associated with training data collection for Doubao and related models. Repeatedly reported by CDNs and site operators as the highest-volume AI crawler on the web and as inconsistent about robots.txt. Blocking it costs you: Little to lose. If you want it gone, expect to block by user-agent at the edge rather than to ask politely in robots.txt. robots.txt token: Bytespider.