The 80,000 visitors a day were one crawler

One day, written up: the traffic on genrewatch was Meta's training crawler, so six sites now sell training bots a one dollar day pass over x402, a residential proxy fleet that no IP block can touch, a Next 16 proxy file that never ran, and why a headless browser does not fix the feed reader.

Read the original (By Anthony Ettinger, via Chovy's Blog): https://dev.profullstack.com/~anthony/blog/040-post.html