After it has been quiet over the last days bots have now started to tackle my forum again and I went from 38% blocking rate of requesting IPs to 76% blocking rate again. Interestingly enough, most of these bots seem not to be recognized by proxycheck.io, despite them having written at the beginning of March that they improved the detection of resident proxies massively:
proxycheck.io blog, read about our latest news.
proxycheck.io
Indeed, they seem to recognize a couple more than before, still just a fraction. If I look on my 24hr statistics of the proxy check API it looks pretty harmless and rather like a good situation: far more "green"visitors than yellow and red and those are during the typical activity times of my forum.
However, if I look at my IP Threat monitor statistics for the same 24 hours the picture changes:
Obviously a lot is blocked that has been let through by proxy check. I can be pretty sure that there are barely any false positives as the number of genuine visitors to my forum is around 700 -1000 in 24h (and this already includes some unidentified baddies) - basically everything on top of that are bad bots. However, it has been worse, looking at the 30 day statistics where we end up with 89% blacklisted IPs:
While proxycheck is incredibly useful it does ignore / fail to detect a lot of the bad traffic - my ASN list and country blocking does the rest. What I see is a subtile change in sources: While the "classics" Brasil, USA, Pakistan, India and Iraq are still on top of the country list the absolute number per country has gone down by at least a third if not more, so it is more distributed now than before. Also, over the last days I had more than once Germany in the top five, which is somewhat unusual, as I only block per ASN and proxycheck here and don't block client providers. And they seem to have somewhat adapted to the daytime of my forums, so the additional traffic does not directly catch suspicion. When looking at the three day statistics of proxycheck one can see the rise within the last 24h...

... while in comparison to the beginning of the month (let alone the peak on March 18th) it is still quiet:
Regarding source ASNs there have been a couple of new kids on the block in terms of datacenter and international distributed ASNs, but not many. I have however seen quite some "old friends" visiting that I hadn't seen in a while including i.e. one of the the Amazon AWS ASNs while the "classics" that knock on the door x times per day stay the same. So it seems that the baddies for once started to do more provider rotating than before and constantly try to find new hosters while on the other hand they do have a reliable set of hosters and ~200 ASNs that either don't care or actively support their dirty business (with some probably being an active core part of it).
Proxycheck does btw. for the most part identify/classify the baddies as VPN, not as proxies. So the scrapers seem to use private VPNs as the basis for their datacenter based attacks to a wide degree (as amount and geographical distribution does it make completely implausible that this would be genuine traffic). Also, I've seen a couple more tor-hosts than usual while I've seen barely any Cable/DSL IP blocked by procycheck.