Crazy amount of guests

[BrettC]

They're just looking for creative ways to thwart your efforts. Don't panic... they'll be back. That test forum, 3 days now, and I just looked at the homepage and it has 1000 online. I think my main site has about 30k online at the moment, but its fine, barely doing much at that level.

These bots are ferocious to say the least. Persistent too.

If you want to humor me a bit, as I'd be curious to see how well the mass-scrapers perform against the Anubis WAF. Even against your load tester, I'd be curious. The botnets seemed to have largely given up on my stuff unless it's a new wave of 'em - then they give up again.

Pre-reqs & Details: https://anubis.techaro.lol/docs/developer/building-anubis/

git clone https://github.com/TecharoHQ/anubis && cd anubis
make deps && make assets && make build

/etc/anubis/instance-yourwebsite.env

BIND_NETWORK=tcp
BIND=0.0.0.0:8923
TARGET=http://your-backend-server-ip:80
POLICY_FNAME=/etc/anubis/policy/botPolicies.yaml
CUSTOM_REAL_IP_HEADER=X-Real-IP
COOKIE_DOMAIN=.yourdomainname.tld
COOKIE_EXPIRATION_TIME=504h
COOKIE_PREFIX=botcheck
COOKIE_SECURE=true
METRICS_BIND=0.0.0.0:9923
METRICS_BIND_NETWORK=tcp
WEBMASTER_EMAIL=some@random-email-address.tld
versionFlag=false
ED25519_PRIVATE_KEY_HEX_FILE=/etc/anubis/key.hex

You can generate a random "ED25519 key" via: openssl rand -hex 32

/etc/anubis/policy/botPolicies.yaml

bots:
  - import: /etc/anubis/crawlers/_allow-good.yaml
  - name: public-paths
    action: ALLOW
    path_regex: ^/(?:\.well-known/.*|favicon\.ico|robots\.txt)$
  - name: lan-whitelist
    action: ALLOW
    remote_addresses:
      - 192.168.20.0/22
      - 10.64.0.0/22
  - name: challenge-all
    action: CHALLENGE
    path_regex: .*
    challenge:
      algorithm: fast
      difficulty: 5
store:
  backend: valkey
  parameters:
    url: "redis://your-redis-server-ip:6379/42"

Adjust the yaml file to your liking. Operates on FIFO.

You can monitor client pass:fails in the redis DB server of your choosing. METRICS_BIND is prometheus supported/connectable. difficulty: 5 is the be-all-end-all check. 0 - no check, 4 - normal quick check, 5 - gets slower, but blocks most bots and then 6+, becomes hell-difficulty, 15 is impossible.

 

[Anthony Parsons]

If you want to humor me a bit, as I'd be curious to see how well the mass-scrapers perform against the Anubis WAF. Even against your load tester, I'd be curious.

Do you just want me to give you root access? As you're familiar with anubis... I have not used it. Its just a test site, there is nothing on it I care about and it will be scrapped in the coming month or two.

 

[BrettC]

Do you just want me to give you root access? As you're familiar with anubis... I have not used it. Its just a test site, there is nothing on it I care about and it will be scrapped in the coming month or two.

You can if you'd like. I don't login with user : password combos, just SSH keys. I'll DM you in a moment.

 

[ES Dev Team]

I'm wondering how this forum still has ~6500 guests while cloudflare's shields are up. it takes 2-3 clicks to start a captcha.
That's kinda concerning because this is what the average of my numbers look like lately.

Could be someone's runnin' some flaresolverr?
What's your guess, people who enjoy using cloudflare?

 

[Suzanne O]

I'm wondering how this forum still has ~6500 guests while cloudflare's shields are up. it takes 2-3 clicks to start a captcha.
That's kinda concerning because this is what the average of my numbers look like lately.

Could be someone's runnin' some flaresolverr?
What's your guess, people who enjoy using cloudflare?

I'd say it's just some malicious bots being annoying that XF staff need to get tougher with and start range banning the bots ip addresses.

 

[BrettC]

I'm wondering how this forum still has ~6500 guests while cloudflare's shields are up. it takes 2-3 clicks to start a captcha.
That's kinda concerning because this is what the average of my numbers look like lately.

Could be someone's runnin' some flaresolverr?
What's your guess, people who enjoy using cloudflare?

Since CF is the #1 free and paid WAF right now, it wouldn't be out of the question if people are creating tools to intentionally evade it.

 

[smallwheels]

Since CF is the #1 free and paid WAF right now, it wouldn't be out of the question if people are creating tools to intentionally evade it.

The downside of a de-facto-monoploly. It's not that this would come in surprising or had not been mentioned as an obvious issue. Also, quite a bunch of the makers of the various scraper solutions and residential proxies advertise with the abilities to bypass CF and turnstile.

 

[BrettC]

Looks like sanity has prevailed on the XF forums...

 

[smallwheels]

Looks like sanity has prevailed on the XF forums...
View attachment 340010

That's pretty crazy - from 200.000 to 61, finally.

day before yesterday at 5 p.m.



at 5:27 (around this time CF "under attack mode" seems to have been set)



and at 5:31:



yesterday at noon:



8 o clock this morning



9 o clock this morning


just now:

 

[BrettC]

That's pretty crazy - from 200.000 to 61, finally.

day before yesterday at 5 p.m.

View attachment 340016

at 5:27 (around this time CF "under attack mode" seems to have been set)

View attachment 340017

and at 5:31:

View attachment 340018

yesterday at noon:

View attachment 340015

8 o clock this morning

View attachment 340014

9 o clock this morning

View attachment 340013
just now:
View attachment 340012

I posted too soon, bots coming back now.

 

[ES Dev Team]

Looks like sanity has prevailed on the XF forums...
View attachment 340010

Wow, record low!
Now, we are back to 1541.

I decided to make my user numbers public so you can rubberneck my Xenforo site that uses fail2ban and a lot of ASN blocks.

 

[BrettC]

Do you just want me to give you root access? As you're familiar with anubis... I have not used it. Its just a test site, there is nothing on it I care about and it will be scrapped in the coming month or two.

For anyone curious, Anubis is now up and running on his test server... tis a funny way of doing it too. Cloudflare filtering -> Anubis filtering -> the website. Abusive bots and scrapers be damned!

 

[Anthony Parsons]

Meaningful impact... compare that to the previously posted 24hr of traffic on that site, with massive spikes, Anubis has certainly regulated things quick smart. For anyone suffering bot craziness, this looks like a meaningful path to possibly explore and experiment with on your own VPS. Good research data for @BrettC too, as it seems dropping a couple million posts onto the internet, bots want that data immediately. This site hasn't even reached a week old, I believe.

 

[BrettC]

Meaningful impact... compare that to the previously posted 24hr of traffic on that site, with massive spikes, Anubis has certainly regulated things quick smart. For anyone suffering bot craziness, this looks like a meaningful path to possibly explore and experiment with on your own VPS. Good research data for @BrettC too, as it seems dropping a couple million posts onto the internet, bots want that data immediately. This site hasn't even reached a week old, I believe.

View attachment 340053

I'd be curious to see the charts now. Because what I am seeing on the console seems to indicate a massive asian-based botnet trying to mass scrape the test forums content... and they're all being brick walled.

 

[Suzanne O]

Had 76 bots on my forum yesterday. Range Banned 3 more IP addresses

 

[Suzanne O]

The only problem i see with all this range banning that i'm doing is many people can't even get on my forum let alone all the bots.
But the thing is if that's the way how hosting is going to be with all the AI bots and the like then maybe we shouldn't be looking to AI at all. We're kind of encouraging the bots to scrape out stuff.

 

[BrettC]

After some more fiddling with Anthony's XF test forum & server with anubis, I'm getting a chuckle out of it. These bots are absolutely going nuts trying to mass scrape what they could see previously and what google has indexed (kinda a key item here)... but are being brick walled at Anubis.

Anubis tonight has been busy throwing out "solve this please" to clients (at one point, was up to 4500 redis keys on a 30 minute window), and they're not even making it past two seconds of remotely attempting to solve it. Some of these scrapers are jumping into the AI LLM honeypot link-hell that Anubis has. That has been greatly entertaining.

As a result, his server is quite literally sleeping right now...


I think the only items on the process list that has been really using any CPU i/o has been ConfigServer & Firewall's LFD log scanner. SQL, nginx, php-fpm, the whole all-in-one-stack is happily humming away. As you all can probably guess, it's the near same toxic IP ranges slamming everyone else's servers.

What I do find interesting on his server is that there is what appears to be some sort of port-exhaustion attack also ongoing to port 443. I've also mostly tackled that issue, and his virtual machine is back to tip-top shape. I do wonder if this IP that his VPS has was some sort of C2 at some point or another, it's the weirdest behavior!

 

[smallwheels]

Because what I am seeing on the console seems to indicate a massive asian-based botnet trying to mass scrape the test forums content... and they're all being brick walled.

After it has been pretty quiet over the last weeks I do see a massive rise over the last two days (on my forum obviously on much smaller scale). The number of requesting IPs per 24h window has roughly doubled while the number of IPs able to pass Threat Manager to actually enter the forum only went up by ~10%, so it is more or less stable and the various existing protection measures and configurations seem to do their job successfully.

Most of the rise in bad IPs seems to come from the US (about 350% compared to usual amount), a much smaller rise (and much lower total numbers) comes from the UK while Vietnam and Brazil (both country blocked) create the usual constant noise of unscuccesful tries (but at the moment even both countries combined way less than half of the currently blocked IPs from the US).

Proxycheck counts all but the country blocks and one can for one see the rise (but with somewhat lower numbers than the actual total) and secondly that proxycheck does not manage to identify the additional traffic as bad:



The number of API requests towards Proxycheck went only up by a good ~30% (way less than the rise in the number of IPs) which indicates that most of the additional traffic is covered by country blocks and already blocked IPs (that have been nasty in the past) and not by ASN blocks (as those would need an API request to proxycheck) or a native Proxycheck rating. As I don't have a country block for the US (but loads of ASN blocks) the US traffic seems for the most part not to be resident proxies (as those would create an API request) but rather data center IPs that have been seen and blocked before (so do not create a new API check). The proxycheck log confirmes that most of the new US traffic in there does come from the usual suspects. So nothing really new and no new players, just someone has intensified his unsuccessful efforts.


PS: The CF Under Attack Mode in these forums is totally annoying: You write a somewhat longer post, try to send it, run into a meaningless generic error message (b/c writing took so long that you have to check the turnstyle chapcha again which XF does not manage to tell you), reload the page, check the turnstile captcha, end again up in your thread and the post you were writing is completely gone. Being a clever guy (and having learned from the past) I do copy the draft before reload now. But honestly: This is embarrassing for XF. I understand the need for protection - but having UAM running for days (after years of neglectence of the bot issue) and hindering regular users from posting and frustrating them by deleting the drafts is a perfect way to kill a community. Not a good show case.

 

[Anthony Parsons]

They seem a little lulled at the moment, on my main site experiment. The door is open right now, and they're sitting on their hands. Slowly climbed from the brick wall 100k daily to what it is now... but still not the million odd daily it was.

 

[ES Dev Team]

Very happy to see these results from anubis. I'm amazed at what can be accomplished with what is mostly a frontend layer.