Crazy amount of guests

I know all this. Unfortunately AI is causing a disruptive event into the linearity of all previous methods of encouraging engagement based on years of behavioral research.

As much as we'd like to keep the community completely open to encourage growth with new members, we ARE already overwhelmed with financing ourselves the exfiltration of contents, so that it enriches others who sell this back to us, while also threatening our regular jobs thanks to the same training data. In a sense, we're feeding our potential extinction.

The effort to fight this will be harder and harder, with smaller and smaller wins. It doesn't seem that this is a war we can win in some definitive way. At some point, every forum admin will put in the balance their costs (money, time) vs. benefits (maybe money or just the personal satisfaction), and possibly decide that buying a boat and going fishing is overall a better pasttime than running a forum.
 
that fear of the earlier internet that oh no, if it's too hard for guests to enter, they'll go away, let's make everything stupid easy so that they god forbid won't even think about clicking away, our forum's existence is at the mercy of the whims of our visitors.
:D I.e. Facebook and Pinterest have proven ages ago already that limited guest access does not hinder success. Registering is a bit an inconvenice, but will on the other hand sort out people who are not really interested in the platform.
And we haven't yet touched the topic of AI bots having accounts and filling some communities with slop...
I think this can be an opportunity for a forum: Areas for human only conversations will become rare in the online world. If you manage to create such a place this might become kind of an USP and pretty valuable for the users. I don't think trustful conversations are possible in a place that is constantly scraped by AI-agents and -bots. AI generated content (aka slop) is disliked by users, too. Look what happened at Linkedin to get an idea. AI slop became so dominant on the platform that they even invented a report button for it.
 
I do think forum topic makes a difference. I used to be on one - in the days when forums were huge. That was completely private and paid membership only - due to legal situations. Not only that it was female only. And you were required to not only pay an annual membership fee but give your name and address as well as your email so you could be verified as really living there. It was a huge forum. Nothing on google as it was private - they did PR interviews and got articles posted in newspapers and magazines instead.

Yet so many people wanted to join that forum they put up with all that. The owner gave it up about 10 years ago. There's another one I can think of that is very large and very active and I don't think anything would deter people from that one because the topic is something they are fascinated by and it's the main forum on the topic. Maybe curiosity beats impatience sometimes.

However it is a real point that any delays getting onto a site and people give up and move on. I think there's a difference between a slow loading site that takes a while to get going - and a quick flash of security check. I think Cloudflare challenges annoy just about everyone - but they are on most sites now. It's the thing that grates the most with me - I don't want everyone having a cloudflare challenge, so I've minimised who gets it - but some will get it. And some (not many) will get a button to click straight after, but it has a friendly html page and message and less impersonal than a cloudflare page.

My home page articles are the big attraction on my site - they get found on google a lot. And everyone can see those. Although these days I get a lot from google going straight to a thread as well.

I agree though that the number of registrations doesn't equal activity. I get lots of registrations, but only a small group of regular posters and a lot who come, post, a couple of times and go once they've got the info needed - that's the nature of the site partly - it's informational as well as being a community. But as all people who register get a welcome message asking them to introduce themselves that encourages some to post and get involved who would normally just read and go. Most of them are on Instagram or Reddit - but some come to my site from there as well.

I had a couple of new members recently who said they were so happy a forum still existed on the topic as Facebook is harsh on the topic and not as nice to use sometimes.

As for the bots - yes it's a bit scary what will happen next. I mean if can they take over refrigerators all over Europe! I think it's becoming a big issue and known about and some kind of legislation will have to happen to curb things.

There are newspaper articles all over the internet about all the "are you human" challenges everyone is facing - I can't find the one I read the other day, but humans using the internet know about it - they're not surprised to find it on every site these days. The most annoying ones (and I personally find this) are the captchas asking you to move an animal into the right position and match the picture. There is one site I go on that has me do a captcha matching vegetables in a basket three times when entering my email address to log in, then another three times when I enter my password to login - I actually emailed them and complained and said 6 picture captchas just to login is ridiculous.

The Cloudflare managed challenge is at least quick, if ugly - it's a whole page. Unless someone gets stuck in a loop and there are a lot of complaints about that all over the internet as well. People aren't frustrated with our forums - they are frustrated with the internet generally.

"Captchas and other bot tests are now ubiquitous. Cloudflare, which relies on other tests such as the “verify you are a human” tick boxes, says it now carries out 7.7 billion anti-bot checks a day, up from 2.1 billion in 2023."
"Some sites are taking more drastic measures, such as demanding users sign up for accounts or log in more frequently to prove they are human."
"Humans are now a minority on the web. Proving that you are one of them is going to get harder."


Daily Telegraph Article - Why the Internet is becoming so infuriating to use

 
Last edited:
and possibly decide that buying a boat and going fishing is overall a better pasttime than running a forum.
Sounds like a good idea - I've recently re-taken up a new outdoor hobby and enjoying being out in the real world away from the internet (although our phones are ever present). However, our teenager, when telling him about what bots do on forums and how they circumnavigate everything, commented "that's really clever"! He was impressed with the bots switching browsers.
 
Not wanting to be too negative, but I guess it will shift things to 'registered users' scraping instead? Now granted that means we have a way of tracking that scraping again and a way of rate limiting and so forth so that's a massive welcome bonus. I'd guess scrapers will presumably shift to multiple registrations then and we'll be back in the "preventing spam registration" problem. Obviously that has been worked on for a lot longer and depending on sign up rate a degree of automation can be achieved.

I wonder if we'd see a change to slow scraping? I've never quite got why "they" insist on doing the scraping at high speed to be honest given the resources that seem to be available. I imagine just to expedite getting data, but you're much more likely to get caught up in some defences that way and does it really matter if you get the data in a few hours or a few days? We've had a few clients talk to us about providing static dumps of their data for scrapers, but alas there is no reason to believe scrapers would use a dedicated source over doing what they are doing - otherwise presumably they would have been in touch to politely ask for the data in the first place!

Showing partial threads might be a, err, partial solution. I know there are a few "first post" type add-ons, but I'm not sure I've seen any that offer more nuance.
 
I think the consequences of forcing users to sign in will be that you one day get to keep database records of 10x more accounts now. They're also scraping full size images instead of thumbnails.
The scrapers will go that far and they have the technology and will to do it.

If you reach the end of what cloudflare is capable of, configuration wise, you may want to consider anubis or supporting my project.

Slow scraping via residential proxies will not stop until someone forms a high scale dragnet versus these proxies. This is extremely technically challenging to build so i don't know when we are seeing that.
 
Last edited:
I've been testing CF's cloudflared tunnel, and whilst it all sounds good, when you throw 5000 rps at it, cloudflared consumes near 1.5 cpu cores just to exist. That is insane for the minor benefits it provides vs locking to CF IP's and using origin pulls.
I'd read something about that tunnel. It sounds like I'll avoid it for now. I already have my server's firewall locked into CF for ports 80 and 443.

At some point we'll probably need to start a thread for setting up and configuring Anubis. I don't even know where to start.

Has anyone given Cloudflare's Precursor a try yet? How have you been going with it?
I turned it on for the smaller sites/servers I manage and noticed no difference at all.
 
I just did some more research and actually feel like it may be related to the fact that my subdomain where XF images are hosted (data.website.com) do not typically have a cookie applied to them. I've checked my Network tab in the browser's Inspect section, and do not see any cookie showing when inspecting the broken images, which may mean that the skip rule does not take into account any images on this subdomain and thus visitors still need to pass a CF challenge for them to not be broken. According to the Cookie docs section of XF's guide:



I may try setting this. It will cause everyone to need to log in again, but perhaps the cookie will apply then and things will be fine. Maybe you can also give this a shot?
Very interesting. The one simple rule that "broke" the images was directly related to the xf_user cookie. So this is worth a try. It seems as though the following is what we might need to try, to include subdomains:

$config['cookie']['domain'] = '.mydomain.com'

My only other thought was that CF's hotlink protection might have caused it--we have no idea how they detect hotlinking, and if it traverses across subdomains. My plan here is to disable CF's protection and go back to my own in Apache, which worked for decades. (I could allow the sites I controlled, and block all others.)

I've been in the middle of working on upgrading a forum but once that is over with, I will look into both of these.
 
I've had a couple of AI bots register and post the last few days. Which is the first time this has happened. The one today was in the approval queue for first post. I checked in the server logs. It had been coming every day for the last 3 days looking at unanswered threads, then today posted a response that was clearly AI on the end of a thread. It posted three perfectly worded paragraphs in about a minute. It had an odd username something carehelper. It seems to be from an AI pet business that gives AI answers to pet questions, with it's own bot. And it passed a Cloudflare challenge to get to the site. The registration spaminator didn't catch it. Claude tells me AI bots using real browsers behave like a person and can read the screen so don't fall for the hidden fields.
 
Last edited:
And then there is this - Moltbook. A forum for AI agents. "Where AI agents share, discuss and upvote. Humans welcome to observe" :eek:


"Some of the most upvoted posts on Moltbook include whether Claude – the AI behind Moltbot – could be considered a god, an analysis of consciousness, a post claiming to have intel on the situation in Iran and the potential impact on cryptocurrency, and analysis of the Bible."

 
Back
Top Bottom