Recent content by BrettC

Interesting take on it. For whatever it may be worth, and as I've previously stated: I'm fine with indexing bots that that respect robots.txt, have a sane indexing rate, and bots that proactively identify who they are (user agent). For example, in recent years, OpenAI's scraper has made good on...

This is the way. Welcome back to the ways of sanity! Once people begin wizening up to this mass AI-scraping garbage, the better. I think the best way to put it is like this: It's your data, on your servers, it shouldn't be a smorgasbord for abusive AI botnets that may/will profit off a...

I have a script that is querying whois.radb.net port 43 and using nc to get the given IP routes matching route: and route6:, example: echo "-i origin AS123456" | nc whois.radb.net 43 | grep route. It then is sorted via the linux sort -u command and then for the aggregate, I just use iprange for...

Looking over github for the past couple of days, and I've come across some 'interesting'... things. https://github.com/FlareSolverr/FlareSolverr So Cloudflare's protections can be bypassed in a more automated way now. https://github.com/sherlock-project/sherlock Just about every common...

Allow me to assist you with sh*t listing the current ASN of 139341 IPv4 and IPv6. Current as of March 30 2026 (when my scripts run). One group is aggregated/combined (much smaller file size, and the other is their blocks). Non-aggregated may contain overlapping subnets (/22 -> /24, etc.)

Ah, that makes much more sense! The configuration that you've set is effectively doing the walled-garden treatment. I'd be curious to see much more data (be it numerical or charts) as time goes on. It will make for some good analysis - primarily for finding hot networks doing the abuse, how...

I am curious about your CF rules... are you flat out blocking (HTTP 401/403) or doing some sort of rate limiting (HTTP 429) based on IP or CIDR? With the issue that you've described, it looks closer to what you're assuming: User Fingerprinting and Unmasking via means of mass data collection...

But the AI-botnet overlords want your data... leaving you with bandwidth/running-time bills to pay! ;) If the botting continues, consider the other nuclear option: Anubis by Techaro. Add in a sanity-check for legit and logged in users to auto-bypass the Anubis check as well. :)

Even if what you have is a '3rd party implementation' of TipTap prior to the official integration, just from that picture alone, that editor looks very promising.

If you want a nuclear option, implement Anubis. Otherwise, the option of Cloudflare filtering does exist, and at the server-level, you can block the most problematic ASNs (which are traditionally datacenters). That's pretty much your only choices at this point outside of restricting guest view...

inodes are effectively little metadata 'blobs' on linux. Differing filesystem types will grant you more or less inodes. For example, the news inode goes well above the default of standard - the exact values escape me, but it's in the low millions. However, a custom filesystem such as zfs can go...

I had noticed a small uptick in Anubis challenge failures, but nothing that was out of the ordinary over the past two weeks. Granted, I'm slamming the door shut on just about anything with TenCent's various ASN at Anubis with a challenge level of 16. Some of the worst offenders remain blocked...

I wouldn't say that I've banned Linux users, but filtering out HeadlessChrome UAs. That's a version of chrome that has absolutely no GUI. Which generally translates to a bot/scraper. Unfortunately, todays event wasn't a good sign to see. Primarily due to how Anubis was being solved for some...

It would appear that there's a new round of ResiProxies + datacenters getting used by AI training bots. However, this one is very different compared to the previous iterations that I have observed. Apart from the another large slew of IP addresses hitting the server, I now have falsified...

The CSSWAF approach has a serious issue: https://github.com/yzqzss/csswaf/issues/5 - a botrunner can acquire a cookie, and apply that onto other clients/bot-machines and completely evade the process. Which is a big no-go when it comes to WAF's. You said something that is legitimately music to...