They also scan mediawiki-projects. Same patterns.
Crazy amount of guests
- Thread starter Levina
- Start date
-
- Tags
- bot swarms guests
guesses:I am wondering: How come that after only a couple of days you see such excessive traffic on a test forum?
- not picked up by the scanners yet
- may have small amount of content ( less to scrape and find )
Man, i had a hard time keeping that one online. We had to build our own knowledgebase software that could stand what the internet could dish out to a 4 core system.
The error log on the test forum showed they were also scanning for sensitive and vulnerable files, exploits too.
I chucked a most visitors online addon on the test forum, for quick reference.
I chucked a most visitors online addon on the test forum, for quick reference.
If you're bored enough or have enough spare time at some point or another, you should consider accepting those POST requests by those
Go-http-client probers, and dumping their post content into a file somewhere. Those IP addresses spanning the same UA and action performed are exceptionally similar to what I had hitting my stuff a long while back in 2025. I just throw HTTP 444 at them now and let the firewall deal with them. A large part of me thinks those things are part of a C2 trying to phone home.So here goes...
That
168.144.110.217 IP is a generic prober, and 151.243.143.47 is abusing all sorts of random User Agents. Unsure if 151.243.143.47 is someone proactively testing or not. 45.88.138.44 is an exploit-check / path traversal exploiter. Again, if you're ever bored, you can intentionally setup some honey pots with redirecting these silly requests to a 'fake file'.On the datemarked log file, the first AI-scraper that I see on the log is:
43.160.241.129 - - [13/Jun/2026:09:15:58 +0000] "GET /members/profilingpete.100/ HTTP/2.0" 200 8315 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1" The second hit is
43.166.244.66 - - [13/Jun/2026:09:27:24 +0000] "GET /misc/style-variation?variation=alternate&t=1781339199%2C076204c8b0b3299ed793d53767021eda HTTP/2.0" 400 6771 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1"And the third appears to be
43.159.152.4 - - [13/Jun/2026:09:37:19 +0000] "GET /api/oembed/?url=https%3A%2F%2Fxfforumtest.com%2Fthreads%2Fthe-community-has-really-helped-me-improve.14073%2F HTTP/2.0" 200 271 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1"I just cannot get over how aggressive the OpenAI / GPTBot is on the logs. That bot is chaining requests! Reminds me of Google Bot back in the early 2000s.
Anyhow... this looks to be an amazon scraper, and the second may be a VPN:
Code:
3.90.72.181 - - [13/Jun/2026:09:43:55 +0000] "GET / HTTP/2.0" 200 13729 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.97 Safari/537.36"
205.169.39.26 - - [13/Jun/2026:09:44:10 +0000] "GET / HTTP/2.0" 200 13729 "-" "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"This one is a google cloud IP:
34.122.147.229 - - [13/Jun/2026:09:45:46 +0000] "GET / HTTP/2.0" 200 13732 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36" and has me a bit sus.Fake referrer:
205.169.39.21 - - [13/Jun/2026:09:47:24 +0000] "GET / HTTP/2.0" 200 13729 "https://bing.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.5938.132 Safari/537.36" - likely bad bot, same actions as the google cloud IP.It's hard to tell, but it would appear that
192.175.111.244 - - [13/Jun/2026:11:01:06 +0000] "GET / HTTP/2.0" 200 13647 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36" is a resi-proxy. After that, things began to pickup with bot scraping. Especially with asian-based IP networks.At
31.125.103.69 - - [13/Jun/2026:18:05:06 +0000] "GET / HTTP/2.0" 200 13650 "https://xenforo.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.4 Safari/605.1.15" onwards is that massive AI scraper botnet.Then a wild google bot pops up here:
66.249.78.37 - - [14/Jun/2026:06:10:38 +0000] "GET /forums/announcements-1/ HTTP/2.0" 200 13894 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.7778.96 Mobile Safari/537.36 (compatible; GoogleOther)". Which pops up during all of that scraping by the AI botnet six hours later... If that isn't sus.And then jumping further down the list
2402:1f00:8000:800::31b2 or to be precise 2402:1f00:8000::/40 is a massive OVH IP range for their Asian DCs. I've seen this IPv6 range slam my servers in months past, only to be rejected at Anubis with a difficulty of 16 'impossible AES256 puzzle' to solve. 
107.172.195.199 appears to be a new one, not seen this particular IP (or webhost/ASN) in use: 107.172.0.0/14 - HostPapa. And the same applies for 154.28.229.167 - non-normal user page-traversal.Above all else, I am seeing A LOT of requests to
/misc/style-variation before mass GET's take place from various IP blocks. Which would be safe to assume there is some new scripting / scraper code going around to target discussion boards, XF included.
Now, looking at the
If what I am seeing is right, the following being the top accessed before 'spidering outward' to the threads:
I may have errors in this, but by total accesses...
And a pretty chart for good measure...
@Anthony Parsons I assume that
access.log file... it is clear that rate limiting by singular IP is not going to cut it anymore. Even CIDR blocks of /24 on IPv4 and /56-ish on IPv6 rate limiting might not be adequate enough. It is so distributed that it would require a hard-line filter/WAF to halt, or in the least, filter, what I am seeing.If what I am seeing is right, the following being the top accessed before 'spidering outward' to the threads:
Code:
23603 /search/
13144 /
11524 /whats-new/posts/
11501 /find-new/posts
4773 /api/oembed/I may have errors in this, but by total accesses...
Code:
224758 103.75.134.0/24
12448 74.7.241.0/24
5190 66.249.64.0/24
3645 66.249.70.0/24
1018 88.99.244.0/24
978 45.88.138.0/24
313 107.172.195.0/24
266 154.28.229.0/24
225 65.21.124.0/24
146 66.249.78.0/24
129 14.191.162.0/24
128 14.191.68.0/24
120 35.252.119.0/24
119 14.191.196.0/24
115 34.19.104.0/24
105 14.191.166.0/24
100 14.191.164.0/24
100 14.191.64.0/24
98 168.144.110.0/24
97 14.191.167.0/24And a pretty chart for good measure...
@Anthony Parsons I assume that
103.75.134.0/24 (103.75.134.239) is something originating from your servers? Grafana k6/2.0.0-rc1 would indicate such.
Yup... everything else... nope. 18k+ uniques... whilst mine was testing, but WOW... a lot for a new site. You found some interesting abuse on what is a brand new forum, what... two or three days old.
Ok, moving on, as I disabled ES on 1 core to remove memory pressure, topped up threads to 10k posts to 200k, did a 150 & 200 VU/ps test, charted, and really, 100 user per second is about the max you would want to hit it with, somewhere soon after that it degrades gracefully into an annoyingly slow site. Stepping it up to 2 core 4gb ram, which honestly, I feel is going to comfortably do two million + posts under significant load as ES should run just fine.
Cleaned up a bit of recent anubis hits so that the following isn't as noisy.
All RU based scrapers... at the same time, hitting two domains that I host for a client - both names redacted. Didn't even get past Anubis tonight.
Code:
Jun 16 00:54:26 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:26.95960894-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"188.72.107.204","x-real-ip":"188.72.107.204","challenge":"019ecefe-d20f-7025-85f5-a2617a9c2a7b","weight":0}
Jun 16 00:54:26 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:26.983089197-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"37.230.196.62","x-real-ip":"37.230.196.62","challenge":"019ecefe-d226-7560-a9d0-22d98d0cd23c","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.197061031-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"45.9.27.28","x-real-ip":"45.9.27.28","challenge":"019ecefe-d2fc-77a4-ae7b-9756d7dd0273","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.347372796-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"188.72.107.204","x-real-ip":"188.72.107.204","challenge":"019ecefe-d392-7d0c-a938-5c11a5654427","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.34984336-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"87.242.123.79","x-real-ip":"87.242.123.79","challenge":"019ecefe-d395-73b0-9cb9-f1135dc62517","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.35021453-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"87.242.123.79","x-real-ip":"87.242.123.79","challenge":"019ecefe-d395-7d6d-84aa-395001eedf44","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.355078958-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"45.9.27.28","x-real-ip":"45.9.27.28","challenge":"019ecefe-d39a-7ab1-b003-fb912dad9a42","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.364090752-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"87.242.123.79","x-real-ip":"87.242.123.79","challenge":"019ecefe-d3a3-72a3-8800-b995f64d241b","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.410557665-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"87.242.123.79","x-real-ip":"87.242.123.79","challenge":"019ecefe-d3d2-73f3-9d38-741929fee870","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.482488147-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"45.9.27.28","x-real-ip":"45.9.27.28","challenge":"019ecefe-d41a-71f9-8288-91ff7ebcab89","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.530609732-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"178.170.196.130","x-real-ip":"178.170.196.130","challenge":"019ecefe-d44a-733d-b81d-ec045c845d6e","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.531209343-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.com","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"37.230.196.62","x-real-ip":"37.230.196.62","challenge":"019ecefe-d44a-7d3e-974f-6847c3286fe1","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.611385513-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"87.242.123.79","x-real-ip":"87.242.123.79","challenge":"019ecefe-d49a-76b3-8e5a-528ddcdd5a84","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.723271263-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"87.242.123.79","x-real-ip":"87.242.123.79","challenge":"019ecefe-d50a-77a5-9bb2-2ad1ea3bc054","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.743436994-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"45.9.27.28","x-real-ip":"45.9.27.28","challenge":"019ecefe-d51e-7c4b-8bad-b6c072eca12e","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.750608089-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"37.230.196.62","x-real-ip":"37.230.196.62","challenge":"019ecefe-d526-730b-89fb-dc49212636c3","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.774522193-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"188.72.107.204","x-real-ip":"188.72.107.204","challenge":"019ecefe-d53e-715c-8251-c5d533979c47","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.910793836-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"188.72.107.204","x-real-ip":"188.72.107.204","challenge":"019ecefe-d5c6-7535-bbdb-f71e99b23334","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.931190287-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"37.230.196.62","x-real-ip":"37.230.196.62","challenge":"019ecefe-d5da-7ac7-bb86-9acd0ba481ee","weight":0}
Jun 16 00:54:27 vm-anubis anubis[686226]: {"time":"2026-06-16T00:54:27.931249388-05:00","level":"INFO","source":{},"msg":"new challenge issued","subsystem":"anubis","host":"<redacted>.mba","method":"GET","path":"/","user_agent":"Mozilla/5.0","accept_language":"","priority":"","x-forwarded-for":"178.170.196.130","x-real-ip":"178.170.196.130","challenge":"019ecefe-d5da-7ad3-bdf9-87e22fcd2ee5","weight":0}AS14061 (Digital Ocean) Singapore
AS400810 (BreezeHost) Singapore
AS213737 (Ayosoft Ltd)
AS132203 (Tencent) Singapore
AS133206 (DGTEK PTY LTD) Australia
AS8075 (Microsoft Corporation)
AS15169 (Google LLC) -> Googlebot
AS24940 (Hetzner Online GmbH) Germany
AS213737 (Ayosoft Ltd) Ukraine
AS36352 (HostPapa) US
AS46261 (QuickPacket, LLC) US
AS24940 (Hetzner Online GmbH) Germany
AS15169 (Google LLC) -> Googlebot
AS45899 (VNPT Corp) Vietnam
AS396982 (Google LLC) US Google Cloud
AS45899 (VNPT Corp) Vietnam
AS396982 (Google LLC) US Google Cloud
AS45899 (VNPT Corp) Vietnam
AS14061 (DigitalOcean, LLC) Singapore
AS45899 (VNPT Corp) Vietnam
Apart from the Australian and the Ukrainian one all well known and apart from the Googlebot-ASN worth blocking the ASN (or, in case of Vietnam and Singapore, the country). In fact apart from the two I had all those already blocked. So still quite happy with my ASN and country blocking list.
Updated the test to 2 cpu 4gb ram, added more threads and posts, but more importantly ran 100vu, then optimised, ran it again, further improvement.
For those with smaller sites, this test replicates 1,000,000 unique users per day hitting your server. 2 cores and 4gb ram is much better than what 1&2 achieved earlier, which was pushing something small and cheap for the sake of experimentation. Realistically, if you're getting 1M uniques a day, then the basic setup is a well optimised 2 cpu and 4gb ram VPS. Even if your site was image intense, those images would all be cached at the edge for users. Its really only addons that may screw you, really at any level, depending on its coding and purpose. Some things are just intensive, regardless how well they're coded.
$25 a month, or less if you find something cheaper, you have full control over your site and server up-to a million uniques daily hitting you. It isn't about how many posts you have, its about how popular your site is.
So the above posts with the smaller VPS which did handle 1M uniques, but it suffered and isn't realistic, yet for every small forum, it would handle this bot craziness with ease for starter forums probably up-to 500K uniques daily without noticeable impact browsing the site. The problem though is bots often hit in a 1 hour window or such, which can change things. Yet this is all without any mitigation strategies in place, this is just balls to the wall open website, no restrictions. Add blocking nasty actors and a small server at $10 a month would be fine with optimisation.
You don't need expensive, you just need access and optimisation. Combined with security against nasty actors - you will be good.
I will hit 2cpu and 4gb ram with more than 100vu, just to test, and will post comparison below to see how far you can push it before degradation and user experience suffers.
Before optimisation was using the optimised 1 core setup. The 2 core allowed further changes since traffic for the past 24hrs highlighted issues in slow queries and mysqltuner.
For those with smaller sites, this test replicates 1,000,000 unique users per day hitting your server. 2 cores and 4gb ram is much better than what 1&2 achieved earlier, which was pushing something small and cheap for the sake of experimentation. Realistically, if you're getting 1M uniques a day, then the basic setup is a well optimised 2 cpu and 4gb ram VPS. Even if your site was image intense, those images would all be cached at the edge for users. Its really only addons that may screw you, really at any level, depending on its coding and purpose. Some things are just intensive, regardless how well they're coded.
$25 a month, or less if you find something cheaper, you have full control over your site and server up-to a million uniques daily hitting you. It isn't about how many posts you have, its about how popular your site is.
So the above posts with the smaller VPS which did handle 1M uniques, but it suffered and isn't realistic, yet for every small forum, it would handle this bot craziness with ease for starter forums probably up-to 500K uniques daily without noticeable impact browsing the site. The problem though is bots often hit in a 1 hour window or such, which can change things. Yet this is all without any mitigation strategies in place, this is just balls to the wall open website, no restrictions. Add blocking nasty actors and a small server at $10 a month would be fine with optimisation.
You don't need expensive, you just need access and optimisation. Combined with security against nasty actors - you will be good.
I will hit 2cpu and 4gb ram with more than 100vu, just to test, and will post comparison below to see how far you can push it before degradation and user experience suffers.
Before optimisation was using the optimised 1 core setup. The 2 core allowed further changes since traffic for the past 24hrs highlighted issues in slow queries and mysqltuner.
Today, numbers have gone up again (I think they were down to something like 2.500):
Annoyingly, at every visit of the forum I get the captcha and even when writing a post and finally sending it I end up with a failure notice:
This can be cured by a reload of the page (captcha again) but in my case parts of what was written before were lost and had to be written again. Not something I would want to have on my own forum.
Yer, its pissing me off. They need to stop the under attack mode and just take some action, either improve resources or block nasty stuff.
Ran a few tests... 1M, 2M, 3M simulated daily uniques. 1M is reasonably handled by a 2 cpu 4gb ram server running XF with ES and optimised. Added: after fixing an issue I found with opcache, then more tuning, even 2M daily unique load is snappy to view under that load.
I'm going to populate the site to a million posts now and leave it sit for a month or two to see how natural bot swarms attack it, how it handles it.
Whilst I am using Linode for this test, which is expensive, for those running smaller sites, you can get a lot for a little. I would personally give OVH or INET a go. https://us.ovhcloud.com/vps/ have 4 cpu 8gb ram under $10 monthly. https://inet.ws/ is more expensive than OVH, but cheaper than Linode or DigitalOcean, choose your location. Then you have https://contabo.com/en/vps/ which is often used for spam... super cheap, but be careful. If you're paying $10 a month for shared managed hosting, you may as well go unmanaged VPS where you can learn, fully control it, ask for help, use far superior setup than shared hosting can provide.
Removed results, running them again, I screwed up with some previous changes, did some more tuning, getting much better results now, will post when done.
Previous results I removed below, as I put some php tuning into backup and forgot to put it back... but the above is a combination of further fpm, opcache and mysql tuning. Big difference between above and below.
My biggest bottleneck was MySQL, so I employed:
I'm going to populate the site to a million posts now and leave it sit for a month or two to see how natural bot swarms attack it, how it handles it.
Whilst I am using Linode for this test, which is expensive, for those running smaller sites, you can get a lot for a little. I would personally give OVH or INET a go. https://us.ovhcloud.com/vps/ have 4 cpu 8gb ram under $10 monthly. https://inet.ws/ is more expensive than OVH, but cheaper than Linode or DigitalOcean, choose your location. Then you have https://contabo.com/en/vps/ which is often used for spam... super cheap, but be careful. If you're paying $10 a month for shared managed hosting, you may as well go unmanaged VPS where you can learn, fully control it, ask for help, use far superior setup than shared hosting can provide.
Removed results, running them again, I screwed up with some previous changes, did some more tuning, getting much better results now, will post when done.
Previous results I removed below, as I put some php tuning into backup and forgot to put it back... but the above is a combination of further fpm, opcache and mysql tuning. Big difference between above and below.
My biggest bottleneck was MySQL, so I employed:
Code:
thread_handling=pool-of-threads
thread_pool_size=2
Last edited:
I've also seen a big increase in bot traffic over the last day or so.
Looking at the web server logs it looks like they have a new strategy of requesting threads by thread number, e.g.
They might just be incrementing the thread numbers, haven't checked that.
This can be picked up easily enough by any anti-bot measures looking at the request path (assuming you have SEO-Friendly URLs enabled).
Looking at the web server logs it looks like they have a new strategy of requesting threads by thread number, e.g.
GET /forums/threads/385472/ (note the lack of thread title)They might just be incrementing the thread numbers, haven't checked that.
This can be picked up easily enough by any anti-bot measures looking at the request path (assuming you have SEO-Friendly URLs enabled).
Just to demonstrate how much MySQL is likely bottlenecking your site, as bot swarms, that is the idea... and this may help you handle it better. This 3M daily uniques rough equivalent, and that small server is completely usable, fast and snappy, even with that load on it.
For me, the last weeks have been quiet: On average ~1600 IPs per day reach IP Threat Manager after passing the .htaccess filtering, ~700 pass IP Threat Manager, ~350 of them are registered users, the rest ist genuine guests, search engines and a couple of resident proxies (which then are treated by Bot Guard). The numbers have been pretty consistent for weeks now, at maximum a couple of hundred more per day from time to time.
So either my forum is to boring to scrape or the drastic locking out of loads of notorious ASNs and countries for a long time already has demotivated the scrapers or the regex-based htaccess-rules that filter out the scanning for weaknesses upon other things cuts loads of crap before it reaches the border of the forum and this way it stays invisible.
They're just looking for creative ways to thwart your efforts.
Don't panic... they'll be back. That test forum, 3 days now, and I just looked at the homepage and it has 1000 online. I think my main site has about 30k online at the moment, but its fine, barely doing much at that level.These bots are ferocious to say the least. Persistent too.
