Welcome to Incremental Social! Learn more about this project here! Check out lemmyverse to find more communities to join from here!
elgordio , 2 months ago My CDN bill recently went from about $5 a month to over $200. Turned out it was Tictok’s spider relentlessly scraping the same content over and over again. It was ignoring robots.txt. In the end I just had to ban their user agent in the CDN config.
My CDN bill recently went from about $5 a month to over $200. Turned out it was Tictok’s spider relentlessly scraping the same content over and over again.
It was ignoring robots.txt. In the end I just had to ban their user agent in the CDN config.