Constant outages with xenforo hosting

[allbuffs]

We migrated from self-hosted which we had for years to xenforo hosted, as we are a volunteer site and thought it would be nice to just have it handled.

This was about 2 months ago and since we've had weekly (or more) outages that last 8+ hours. Support is never online due to the time differences, so our server doesn't get restarted until morning London time (we are Denver)

Support provides no updates on a permanent solution, or really any information at all.

Has anyone had xenforo host and how did that go? Right now we are very regretful.

 

[M@rc]

Only way to resolve this once and for all is to submit a support ticket with XF or download a backup through your Admin Panel and move again.

I don't regularly see people post stuff like this but they're probably submitting support tickets (which is all they can really do)

 

[ES Dev Team]

That's unfortunate. From what i understand, they are using cloudflare to protect this site, so it may be the same tech on the cloud version.
I've noticed that this system's ability to defend sites is degrading. I use fail2ban for our self hosted xenforo installation, and get very good results compared. ( as of this moment, Xenforo.com has 12000 guests which are almost all bots, whereas our site has 2500. On both cases, a majority of the guests are AI scraper bots, and we have identically sized amount of content, so the sites are comparable )

Bots have ticked up massively since the start of the year. The larger the site you have, the more bots you have.
This is affecting a lot of websites and very few server administrators have a handle on it.

Hope you get what you need!

 

[Chris D]

Hello @allbuffs

We have been discussing your ticket this morning which we will reply to in due course.

We're making progress with the stability of your site in particular, which seems to be related to memory issues with regards to MariaDB. I will explain more in our ticket. I just wanted to assure you that it's being looked into and being taken seriously.

You will find from most Cloud customers and from the sentiment posted above, this isn't a common issue and we generally pride ourselves on the stability of the platform. This is just "one of those" things that is proving somewhat elusive to resolve, but we are taking it seriously.

Indeed, as you already noted, the issue overnight was not a database issue. It seems to have been related to traffic at that particular moment which then recovered to normal levels. While nothing specific to announce, yet, we're in talks with some new partners to provide much more robust DDoS mitigation and prevention which should alleviate this type of issue. But as noted above, this kind of traffic is, unfortunately, part and parcel of the state of the internet at the moment and it's a bit of a "cat and mouse" situation and web services globally are struggling with the increase in bot traffic and activity.

We can't make specific promises to resolve that directly as it is fairly sporadic and situational and somewhat the "nature of the beast" at the moment, but the intermittent database issue is definitely something we're aiming to resolve sooner rather than later.

I'll be in touch!

Chris

 

[allbuffs]

I appreciate all the responses and PMs. Truly I do.

We are going to disable our adsense stuff. We remove ads for paying subscribers, which is the majority of the users who participate on the site and sometimes forget what the unpaid experience is like. Maybe that will make a difference.

When we self-hosted we would have occasional problems with AI crawlers who would trigger some nginx errors, but nothing on this scale.

We also have a huge amount of spam cleanup in our moderation queue each day. When self-hosted it was 5-6 items a day, now it is closer to 80-90

 

[Chris D]

Just to stress this is just a sign of the times rather than being because you’re on Cloud.

The software is the same regardless with the same anti-spam features. Attracting more bots, AI crawlers, more sophisticated and evolving spam attacks aren’t really impacted by where you’re hosted.

Mitigations vary, of course, for things like DDoS attacks and preventing AI crawlers but best-in-class tools for such things already exist, often free, for users of Cloudflare so that is certainly worthy of a consideration to help, along with our continued efforts battling the issue at the infrastructure level. None of these things really affect spam sign ups or content though. That’s just a necessary evil to battle as it evolves.

 

[allbuffs]

We are looking at both our robots.txt (which didn't carry over when migrating to hosted) and ads.txt - neither of which are operating quite how they should. I do understand that is an industry problem right now. Because of that, what we've seen is:

• Excessive population of ads from Google's AI-driven optimizer, which is serving more ads in a futile effort to capture lost revenu
• Massive deterioration in the quality of our ads, and a disaster in our inventory reputation
• The visitor spikes causing fill rates to be all over, which is impacting real users

No idea if it will help, but we are pausing our ad spend and removing banner ads. We are also updating robots.txt to what we had when self-hosting to see if that assists.

 

[ES Dev Team]

The massive amounts of asian AI scraper bots afflicting today's websites do not follow robots.txt or ads.txt.
They also tend to use a real browser to bypass a number of anti-bot protections. So they will even show up on google analytics and not be filtered out. The same thing could happen with ad networks, where the bots get served ads.

You have the enforce said rules, or your website is toast as of this year.
Here's what a few seconds of my fail2ban protection logs look like:

2025-09-28 00:01:24,286 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 200.107.68.6
2025-09-28 00:01:24,393 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 179.127.194.178
2025-09-28 00:01:24,603 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 138.36.49.101
2025-09-28 00:01:24,738 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.189.64.232
2025-09-28 00:01:24,876 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.163.119.45
2025-09-28 00:01:25,173 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.189.249.76
2025-09-28 00:01:25,622 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 190.45.187.87
2025-09-28 00:01:25,707 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 170.83.133.225
2025-09-28 00:01:25,810 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 177.104.100.220
2025-09-28 00:01:25,986 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 186.232.133.139
2025-09-28 00:01:26,073 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 177.20.183.208
2025-09-28 00:01:26,227 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 138.219.238.45
2025-09-28 00:01:26,369 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.177.140.176
2025-09-28 00:01:26,701 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 41.74.57.207
2025-09-28 00:01:27,041 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 190.92.109.158
2025-09-28 00:01:28,402 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 187.102.0.155
2025-09-28 00:01:29,702 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 187.73.203.47

This system bans thousands of IPs per day at this point.
I manage 30 servers and every server i manage encounters these scraper bots.

You need an added protection layer if Xenforo cloud hosting is not providing one out of the box. The enormous amount of bots have made this basically mandatory for running any website today.

 

[Paul B]

We are looking at both our robots.txt (which didn't carry over when migrating to hosted)

Both robots.txt and ads.txt are managed from within the ACP at /admin.php?options/groups/basicBoard/ on Cloud.

 

[allbuffs]

The massive amounts of asian AI scraper bots afflicting today's websites do not follow robots.txt or ads.txt.
They also tend to use a real browser to bypass a number of anti-bot protections. So they will even show up on google analytics and not be filtered out. The same thing could happen with ad networks, where the bots get served ads.

You have the enforce said rules, or your website is toast as of this year.
Here's what a few seconds of my fail2ban protection logs look like:

2025-09-28 00:01:24,286 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 200.107.68.6
2025-09-28 00:01:24,393 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 179.127.194.178
2025-09-28 00:01:24,603 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 138.36.49.101
2025-09-28 00:01:24,738 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.189.64.232
2025-09-28 00:01:24,876 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.163.119.45
2025-09-28 00:01:25,173 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.189.249.76
2025-09-28 00:01:25,622 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 190.45.187.87
2025-09-28 00:01:25,707 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 170.83.133.225
2025-09-28 00:01:25,810 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 177.104.100.220
2025-09-28 00:01:25,986 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 186.232.133.139
2025-09-28 00:01:26,073 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 177.20.183.208
2025-09-28 00:01:26,227 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 138.219.238.45
2025-09-28 00:01:26,369 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 45.177.140.176
2025-09-28 00:01:26,701 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 41.74.57.207
2025-09-28 00:01:27,041 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 190.92.109.158
2025-09-28 00:01:28,402 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 187.102.0.155
2025-09-28 00:01:29,702 fail2ban.actions        [2340045]: NOTICE  [apache-noheadoldes] Ban 187.73.203.47

This system bans thousands of IPs per day at this point.
I manage 30 servers and every server i manage encounters these scraper bots.

You need an added protection layer if Xenforo cloud hosting is not providing one out of the box. The enormous amount of bots have made this basically mandatory for running any website today.

Yeah - this is precisely what we are seeing. Tons from China and Singapore with real browsers. Thanks for the additional context