Crazy amount of guests

BrettC said:
Considered a secondary nuclear option? Anubis. "Kick'em right in the balls solution" is what Anubis ultimately is. 😁
Click to expand...
Unfortunately I am unsure about this one because I heard it can be quite nuclear and more annoying for more legitimate users, and quite frankly I don’t have enough technical expertise to figure out how to install it. 😅 Though I’m definitely curious. Need to read up on it more.
 
Growlithe said:
Unfortunately I am unsure about this one because I heard it can be quite nuclear and more annoying for more legitimate users, and quite frankly I don’t have enough technical expertise to figure out how to install it. 😅 Though I’m definitely curious. Need to read up on it more.
Click to expand...
Yeah, when i got started with Anubis, it was a bit confusing with how to get it setup and operational with multiple host names and multiple backend servers - that was a steep curve to learn!

For whatever it is worth, Anubis can check for particular cookie existence. So it can be set to bypass or severely lessen the challenge level for legitimate users if they're logged in. Essentially, Anubis is a 'Man In The Middle' between the backend server host and the frontend load balancer or ingest server.

I need to write up a how-to on it at some point. It's really straight forward if you go the route of compiling it from source.
 
Has anyone been dealing with a crazy amount of guests trying to access /showthread.php? We migrated from vBulletin to XF some years ago, so the showthread.php is an important part of redirects for us as a 20+ year old forum. I've had no choice but to block it recently since bots are absolutely pummeling it. We also had traffic from ASN 701 try to DDoS us yesterday. It's been absolute chaos and I am so exhausted.
 
They've given up on me for the time being. I have unchecked every CF bot block there is... so I could get good data... and they gave up instead.
 
Xon said:
I recently had a client having bots requesting a huge number of misc/style links due to all the unique _xfRedirect attribute in the URL. With the link-post.js backport my StandardLib add-on does, this isn't actually required to have the front-end behave sanely for actual users.
Click to expand...
A part of me wonders if the bots are seeing the Redirect portion in the attribute then are attempting to use it as a Cross-Site Scripting (XSS) attack vector, or a blind-redirect. I've not really reviewed the code to see what is going on there, but it sure gives off XSS-attempt vibes by how you describe it.

FWIW: https://forum.domainname.tld/misc/style?style_id=0&_xfRedirect=https%3A%2F%2Fforum.domainname.tld%2F - the _xfRedirect being the core focus here. Correct me if i am wrong here, but I believe _xfRedirect is being defined/utilized within ./src/XF/App.php with the function named getDynamicRedirect for non-Administrative clients.

A quick find+grep (find . -type f -name "*.php" -not -path "*/code_cache/*" -exec grep -H "xfRedirect" {} +) depicts the following for the xfRedirect text-bit:
Code: 
./src/XF/Template/Templater.php:                return '<input type="hidden" name="_xfRedirect" value="' . htmlspecialchars($redirect) . '" />';
./src/XF/Admin/Controller/AddOnController.php:                                  '_xfRedirect' => $this->buildLink('add-ons/install-from-archive-complete', null, ['batch_id' => $batch->batch_id]),
./src/XF/Admin/Controller/AddOnController.php:                                  '_xfRedirect' => $this->buildLink('add-ons/delete-files-complete', null, ['addon_id' => $addOn->getAddOnId(), 'title' => $addOn->title]),
./src/XF/Admin/Controller/TemplateController.php:                       $redirect = $this->buildLink('templates/edit', $template, ['_xfRedirect' => $dynamicRedirect]);
./src/XF/Admin/Controller/ActivitySummaryController.php:                                        '_xfRedirect' => $this->buildLink('activity-summary', null, ['test_email_success' => 1]),
./src/XF/Admin/Controller/ToolsController.php:                                          '_xfRedirect' => $this->buildLink('tools/rebuild', null, ['success' => 1]),
./src/XF/Admin/Controller/ToolsController.php:                                  '_xfRedirect' => $this->buildLink('tools/file-check/results', $fileCheck),
./src/XF/Admin/Controller/OptionController.php:                 '_xfRedirect' => $returnUrl,
./src/XF/Admin/Controller/LoginController.php:                          '_xfRedirect' => $redirect,
./src/XF/Admin/Controller/PhraseController.php:                 $redirect = $this->buildLink('phrases/edit', $phrase, ['_xfRedirect' => $dynamicRedirect]);
./src/XF/Admin/App.php:                                         'admin.php?tools/run-job&only_ids=' . urlencode($onlyIds) . '&_xfRedirect=' . urlencode($url),
./src/XF/Pub/Controller/SecurityLockController.php:                                     '_xfRedirect' => $this->buildLink('index'),
./src/XF/Pub/Controller/RegisterController.php:                                 '_xfRedirect' => $redirect,
./src/XF/Pub/Controller/AbstractController.php:                         '_xfRedirect' => $this->request->getFullRequestUri(),
./src/XF/Pub/Controller/AbstractController.php:                                         '_xfRedirect' => $this->request->getFullRequestUri(),
./src/XF/Pub/Controller/AbstractController.php:                         '_xfRedirect' => $this->request->getFullRequestUri(),
./src/XF/Pub/Controller/AbstractController.php:                         '_xfRedirect' => $this->request->getFullRequestUri(),
./src/XF/Pub/Controller/LoginController.php:                            '_xfRedirect' => $redirect,
./src/XF/Pub/Controller/LostPasswordController.php:                                             '_xfRedirect' => $this->buildLink('index'),
./src/XF/App.php:               $redirect = $request->filter('_xfRedirect', 'str');
 
Last edited:
MapleOne said:
Human verification AndyB

Human verification

Adds a human verification overlay to your forum.

That extension eliminated virtually all robots except for the ones allowed in a few hours.
Click to expand...
Given that the exceptions are simply based on the transmitted user agent name any fake google bots etc. will pass this. Other add ons do in addition to the name use reverse DNS to check the authenticy of search engine crawlers which is clearly way more reliable.
 
smallwheels said:
Given that the exceptions are simply based on the transmitted user agent name any fake google bots etc. will pass this. Other add ons do in addition to the name use reverse DNS to check the authenticy of search engine crawlers which is clearly way more reliable.
Click to expand...

Try it, go to settings, remove the allowed bots and wait a few hours. Most all bots are removed that is all that matters.
 
smallwheels said:
Given that the exceptions are simply based on the transmitted user agent name any fake google bots etc. will pass this. Other add ons do in addition to the name use reverse DNS to check the authenticy of search engine crawlers which is clearly way more reliable.
Click to expand...
Have to agree. The addon has good intentions, but I personally would prefer that sort of verification-check to be 'out-in-front' prior to the actual web server that is serving the PHP and performing SQL queries.

Off-loading/shedding as much as possible is the utmost ideal. For example, static objects served from a CDN, malicious clients sent to a WAF or outright blocked, and pushing abusive scrapers to a WAF, while doing a light-weight check or bypass for legitimate clients, well before hitting the actual server processing the request is the name of the game.

I'm a bit in the boat with @Anthony Parsons in the sense of optimization in that regard.
 
BrettC said:
Off-loading/shedding as much as possible is the utmost ideal. For example, static objects served from a CDN, malicious clients sent to a WAF or outright blocked, and pushing abusive scrapers to a WAF, while doing a light-weight check or bypass for legitimate clients, well before hitting the actual server processing the request is the name of the game.
Click to expand...
I prefer this also--catch all the bad stuff before it hits the server. Anything I do on the server or, worse (resource-wise), in XF using an add-on, should be additional hurdles for the ones that get through.

That is why I'm really liking CF's Zero Trust for logins to XF's admin area, or the install directory, or any other non-public areas--anyone attempting to get in is stopped before they even get to the server.
 
MapleOne said:
Try it, go to settings, remove the allowed bots and wait a few hours. Most all bots are removed that is all that matters.
Click to expand...
The big question is: How do you know/verify that all bots are removed? I won't try it as I have another setup at work tat works pretty well and see no reason to change it.
 
BrettC said:
The addon has good intentions, but I personally would prefer that sort of verification-check to be 'out-in-front' prior to the actual web server that is serving the PHP and performing SQL queries.
Click to expand...
The "how" is independent from the "where" - I do have the reverse DNS verification within two XF add ons and no issues of any kind with that. Surely, if we are talking about massive loads or DDOS you are doomed w/o CF or similar but at normal times the amount of search engine requests to check is low so it is absolutely no issue and I doubt that the typical DDOS attack would hit in form of fake search engine user agents.
 
smallwheels said:
The big question is: How do you know/verify that all bots are removed? I won't try it as I have another setup at work tat works pretty well and see no reason to change it.
Click to expand...

I don't mean to argue with anyone here and I understand we all have different needs, I installed this with the default settings allowing...

applebot,bingbot,googlebot
Click to expand...

It took a few hours for my BOT count to go down to almost nothing other than the 3 bots I allowed above. I was a little annoyed that Apple has so many bots on the site but nonetheless I shaved hundreds of BOTS off my list.

I flick this addon off and on as needed and the Bot count drops significantly every time I activate it.

If you want easy one step installation that will drop the number significantly then this addon does what it says.

For other more advanced Xenforo users I understand there are more complex solutions but this one is easy for a novice user and it does make a real difference.
 
I use HUMAN VERIFICATION (XF2) on one forum and XF BOT GUARD (Zeebot) combined with CLOUDFLARE on another forum.

Both are getting the job done, so far!
 
Saw a big cluster from vultr, looks like bright data is using them. All hostnames are assigned as lum-int.io

What follows is a blocklist in apache .htaccess format. I return 403 for these IPs and fail2ban scoops them up.

Code: 
#Vultr signapore attack 07/21 lum-int.io

Deny from 139.180.128.0/19
Deny from 149.28.136.0/19
Deny from 207.148.126.0/21
Deny from 45.32.99.0/23
Deny from 45.32.100.0/22
Deny from 45.32.103.0/22
Deny from 45.32.104.0/22
Deny from 45.32.105.0/22
Deny from 45.32.106.0/22
Deny from 45.32.107.0/22
Deny from 45.32.108.0/23
Deny from 45.32.109.0/23
Deny from 45.32.110.0/23
Deny from 45.32.111.0/23
Deny from 45.32.112.0/23
Deny from 45.32.113.0/23
Deny from 45.32.114.0/23
Deny from 45.32.115.0/23
Deny from 45.32.116.0/23
Deny from 45.32.117.0/23
Deny from 45.32.118.0/23
Deny from 45.32.119.0/23
Deny from 45.32.121.0/22
Deny from 45.32.122.0/22
Deny from 45.32.123.0/22
Deny from 45.32.124.0/22
Deny from 45.32.125.0/22
Deny from 45.32.126.0/22
Deny from 45.76.144.0/21
Deny from 45.76.145.0/21
Deny from 45.76.147.0/21
Deny from 45.76.148.0/21
Deny from 45.76.150.0/21
Deny from 45.76.152.0/23
Deny from 45.76.153.0/23
Deny from 45.76.155.0/23
Deny from 45.76.156.0/22
Deny from 45.76.157.0/22
Deny from 45.76.160.0/22
Deny from 45.76.161.0/22
Deny from 45.76.162.0/22
Deny from 45.76.163.0/22
Deny from 45.76.176.0/20
Deny from 45.76.177.0/20
Deny from 45.76.178.0/20
Deny from 45.76.179.0/20
Deny from 45.76.180.0/20
Deny from 45.76.181.0/20
Deny from 45.76.182.0/20
Deny from 45.76.183.0/20
Deny from 66.42.48.0/20
Deny from 64.176.0.0/15

before i added this banlist, the bots have been keeping fail2ban busy and hitting record numbers.

1784662620445.webp
 
Happy to hear that for some of you guys they seem to have died down! Our bot attacks from SEA countries have died down a lot, and Indonesia is no longer attacking us, but I have continued to face issues with a huge amount of bot traffic to old vBulletin php pages - specifically /showthread.php, /showpost.php, /blog.php, /member.php, etc. Some of these are very easy to block since they're not useful redirects, but in the case of things like /showthread.php it's a bit more tedious since we have a lot of old links scattered around as a 20+ year old forum. But I've had to block them entirely, otherwise we get thousands of requests within 30 minutes to these pages.
 
You must log in or register to reply here.

Similar threads

O
  • Question Question
XF 2.2 Huge amount of guests
Replies
6
Views
189
infinitysnapz
infinitysnapz
Wildcat Media
Unusually high number of guests (4x to 5x normal)
Replies
8
Views
222
Tumpe
Tumpe
Tyson
Possible to limit amount of threads a guest can view?
Replies
4
Views
1K
Junior
J
Back
Top Bottom