Crazy amount of guests

[Wildcat Media]

They were the best... oh wait... it fights for equal spot with the Telex machine.

True dat.

Although I have to admit, for the time, those Epson dot matrix printers never let us down. And we needed them for multi-part printing. I believe I had three of them set up--one for invoices, another for packing lists, and a third for pressure-sensitive shipping labels. Towards the end, we had a thermal label printer provided to us by the shipping company as we needed to print their barcodes.

Somewhere in my storage bins, I still have a daisy wheel printer. It probably still works, but who knows if I could ever find a ribbon cartridge for it anymore. I hate tossing out stuff like this, but on the other hand it's fairly useless for anything I do now.

 

[Wildcat Media]

Back to our crazy amounts of guests.

I found an issue that was a by-product of limiting one of my smaller forums to human visitors only, by using a managed challenge. I had it set so that logged-in users were whitelisted, and everyone else was issued a managed challenge. No complaints from anyone while it was in place but, again, it's a lower-traffic forum.

The problem, though, was that even with logged in users, images in my R2 bucket (attachments and avatars) were not appearing in threads or on the forum.

Is there something else I should whitelist to prevent that from happening? IP addresses, or Cloudflare's ASN, maybe? The R2 buckets have worked for a few years now with no issue, so it's not a configuration problem.

 

[ES Dev Team]

AFAIK, cloudflare puts a generic rate limiter on everything... i immediately had to whitelist IPs of all our providers after setting it up.

 

[Wildcat Media]

This isn't rate limiting--it's a low-traffic forum. Why would images work for years prior to changing a security rule and stop working after? That's what I'm trying to figure out.

I did whitelist Cloudflare's ASN for now, but I don't know if that's the correct fix.

 

[Rhody]

Right now:

Total: 201 (members: 21, guests: 180)
Bots: 88,124 (members: 0, guests: 88,124)

It's crazy!

 

[puterfixer]

Cloudflare is now in the position to throw its weight around and influence some changes online, and it will become the default rule for filtering in just a few weeks for a good chunk of the internet. They just declared war on AI scrapers without naming them but clearly targeting Google in particular because of its use of the same spiders for search indexing and scraping contents for AI training, which forces content owners into a corner with a difficult decision. The tide is turning on Google from other sides, too; publishers are starting to opt out of any Google indexing. Links below.

https://www.programming-helper.com/tech/cloudflare-ai-crawler-policy-september-2026-blocks-training-agents-default

Publishers Are Preparing to Opt Out of Google Search

The nuclear option is gaining traction as web traffic collapses and Google refuses to negotiate with content creators

www.adweek.com

 

[zeeb0t]

Cloudflare is now in the position to throw its weight around and influence some changes online, and it will become the default rule for filtering in just a few weeks for a good chunk of the internet. They just declared war on AI scrapers without naming them but clearly targeting Google in particular because of its use of the same spiders for search indexing and scraping contents for AI training, which forces content owners into a corner with a difficult decision. The tide is turning on Google from other sides, too; publishers are starting to opt out of any Google indexing. Links below.

https://www.programming-helper.com/tech/cloudflare-ai-crawler-policy-september-2026-blocks-training-agents-default

Publishers Are Preparing to Opt Out of Google Search

The nuclear option is gaining traction as web traffic collapses and Google refuses to negotiate with content creators

www.adweek.com

It is interesting, so soon after announcing their plans for x402 / metered scraping, they are now against AI scrapers in general by default. Money talks.

 

[smallwheels]

The tide is turning on Google from other sides, too; publishers are starting to opt out of any Google indexing.

I think this is a pretty good move. Over the last years, Google has been in a position where no one dared to resist as resistance would have bombed visibility for visitors. This changes and it is good that now Google starts to see borders.

Money talks.

Clearly. But there are three basic, but vastly different possible interpretations:

1. CF are the good guys. Now, that they do have a technical solution, they give website owners the choice.

2. CF use their customer base to act basically as a union for their customers. Self-declared, but the principle is the same: the power of many small, powerless entities combined in one voice to resist the huge, massively powerful opponent, that the small ones are dependent from and that has bee misusing his power more and more.

3. CF have spent years to achieve coming as close to a monopoly in their field as possible, including using the same methods that made Rockefeller once the richest man on earth (give away the lamps for free, sell the oil to make use of them). While pretending to act for the good of all they are clearly a commercial company, selling their better working protection plans for sums that only few can afford, leaving the rest and majority with mediocre protection and providing that at all b/c they need the data of the free clients to improve their paid services and the high number of free clients to gain market power.
Now, serving 20% of the overall internet traffic and 42% of the Fortune 500 companies they feel big and powerful enough to challenge and tackle the biggest giant of the internet: Google. And claim once more their motivation would be altruistic and the good for all. In fact what they may want to achieve is to replace Google as the one powerful company to rule them all - in a bidirectional way: To overcome giants like Google, Bing and Apple so that those in future have to play by the rules that Cloudflare sets while at the same time website owners have to play by their rules as well, become CF customers and, being locked in and dependent from Cloudflare - if they are not able to pay huge sums to CF - live with whatever CF gives them. Their "pay for crawl" model seems pretty similar to what Spotify has done to musicians: Basically impossible to avoid, but you get paid ridiculous pennies for the work you created. Only few will profit or be able to generate a recognizable income. So basically the idea is to replace the Google monopoly with a Cloudflare monopoly and the question is if that would be better or even worse than before.

If you believe in 1, 2 or 3 is up to you.

 

[zeeb0t]

If you believe in 1, 2 or 3 is up to you.

They follow the money, which seemingly - taxing every bot on the planet would give them more of it. So, I'll lock in #3. We will all receive between nothing and a pittance, but their tax on a pittance for almost every website will earn them a pretty penny.

 

[zeeb0t]

They follow the money, which seemingly - taxing every bot on the planet would give them more of it. So, I'll lock in #3. We will all receive between nothing and a pittance, but their tax on a pittance for almost every website will earn them a pretty penny.

I should add - for the bots, legally receiving rights to do whatever they like with said content, for nothing to a pittance, is also a pretty sweet deal. It's a classic bait and switch. Make us scream to the point of desperation, that we accept the desperate solution.

 

[Wildcat Media]

Publishers Are Preparing to Opt Out of Google Search

The nuclear option is gaining traction as web traffic collapses and Google refuses to negotiate with content creators

www.adweek.com

That AdWeek article is chilling in a way. Read between the lines and you see part of the wide path of destruction that AI is causing on so many fronts. (Yet another reason why I absolutely hate AI without reservation, and refuse to use it.)

To sum it up: Search engine referrals drop since people are using AI. And why should people visit a site when they can have AI spit out a (usually incorrect) answer? So we have major networks like USA Today, Newsweek, etc. seeing or expecting drops in visitors (reading between the lines) as the public moves to AI to retrieve the same information, which means declining revenues from having eyeballs on their sites reading articles and clicking on ads.

Extend that to forums. Social media hasn't killed forums yet (and it never will IMHO), but AI just might. AI scrapers will soak up our content without compensation, without credit, and certainly without permission, and spit out that information without them ever lifting a finger to visit our forums. We lose visitors, we lose people who could contribute (they won't even visit the forum), and eventually it's no longer worth running a dead forum with no new contributions.

That's our future. And yet the AI arms race continually hammers on our forums with many using dubious means to get around any anti-scraper protection we have in place. We're just the little guys, up against the oligarchs and corrupt politicians who won't lift a finger--they just pad their shareholders' pockets on our efforts.

 

[Anthony Parsons]

AI scrapers will soak up our content without compensation, without credit, and certainly without permission, and spit out that information without them ever lifting a finger to visit our forums.

I don't know about that. Search engines do that now, they respond with the most accurate information to answer the question, AI just elaborates. But AI still doesn't satisfy by giving an answer, where people want discussion. No different than people find something via search, then choose to move to the site for more, AI is no different in my eyes. I am experiencing more traffic now from AI than I was from Google alone, so I just don't think that's accurate.

 

[zappaDPJ]

AI just elaborates. But AI still doesn't satisfy by giving an answer, where people want discussion.

From a user perspective that elaboration usually gives me the answer I want so I don't go to the source. I used to as a matter of conscience and that in the past might have lead to a registration and engagement but not any more, at least not for me.

There's a lot of debate right now as to whether the value of SEO is being diminished by AI and your increase in traffic is interesting because the general consensus is that AI has resulted in a big rise in 'zero-click' searches.

In general I have to agree with @Wildcat Media. Whatever it is, if it's on the internet it's being hoovered up and spat back out with no regard or compensation for the original source.

 

[Wildcat Media]

But AI still doesn't satisfy by giving an answer, where people want discussion. No different than people find something via search, then choose to move to the site for more, AI is no different in my eyes.

The problem is, many AI models don't quote a source--they just present sentences that the average person glosses over and accepts as fact, without any attribution. They won't know if it came from Reddit, a forum, Wikipedia, a biased blog post on Medium, etc. Someone might look up the topic of my niche forum, see an unattributed answer, and go on with life. They won't bother to visit the forum, or even know it exists. This is how forums will die.

It's like the AI slop we see with Amazon's reviews--they just pick and choose highlights from various reviews, but do not link to a single one of them that mentions what their summaries show. Given how Bezos operates, we can't trust him to provide anything truthful; he just wants a tool to increase sales, and hand-feeding their prepared slop to consumers over reading the actual reviews is a way to do this.

On our forum, in a non-public off-topic area, we have literally dozens upon dozens of examples of how flawed AI's output is, how more and more people are rejecting it (especially Gen Z), and how it is destroying the artistic communities who are vehemently opposed to it, as I am. So...no, I have no need or use for AI in my life, nor do I abide by all the damage their datacenters are doing to our economy and our planet.

 

[zappaDPJ]

and how it is destroying the artistic communities who are vehemently opposed to it

Interesting that you should mention that because it's exactly what's happened in my line of work. I've spent my entire working life working within the music industry, as a composer, performer, producer and engineer, and all of those areas are being decimated by AI at an alarming rate.

 

[Anthony Parsons]

I've spent my entire working life working within the music industry, as a composer, performer, producer and engineer, and all of those areas are being decimated by AI at an alarming rate.

Yer, I gotta admit, I have listened to and watched some of those on youtube... and they're good. Some are friggin amazing.

I am starting to wonder if either CF have rolled something out OR the people behind the bots are having a reset and recalibration for the next round. I have literally every bot blocking setting turned off, zero rules, nothing, and the traffic is primarily legit people and crawlers since last months peak.

 

[zappaDPJ]

Yer, I gotta admit, I have listened to and watched some of those on youtube... and they're good. Some are friggin amazing.

It is indeed unreal. Just to put some perspective on it I've composed and recorded a song about the trials and tribulations of maintaining the steady development of a forum project (no disrespect intended). The time and effort it took to create this masterpiece AI slop took between 5 and 10 seconds.

Thread Lock Day

Thread Lock Day - Song by usera1fc833f8e014ba9 | Treblo

treblo.com

If that was a genuine project, multiple highly skilled people would have been involved and got paid. If you are not scared of where AI is currently heading you really should be.

 

[Rhody]

I don't know about that. Search engines do that now, they respond with the most accurate information to answer the question, AI just elaborates. But AI still doesn't satisfy by giving an answer, where people want discussion. No different than people find something via search, then choose to move to the site for more, AI is no different in my eyes. I am experiencing more traffic now from AI than I was from Google alone, so I just don't think that's accurate.

Agreed. I created the llms.txt file and required AI bots to cite us as a source (with the text I specified in the link). We are actually getting a lot of new forum members, from being cited as a trusted source of information on the topic.

But they ARE out of control, too. Even with Cloudflare and XF Bot, I am currently at 38,876 bots. (down from 88,000) With the two above tools, it is no longer crashing my sql server, but that is still insane.

I think many of them are malicious bots. Many seem to be targeting a specific post in a specific thread, that is in a private subforum of the forum. (It requires account/login to view or access that section). I since deleted that thread, but they are still aggressively trying to open it. How they even knew about it is a mystery.

 

[Greenandgold]

I addressed this issue on my site when I had a massive issue a while back https://xenforo.com/community/threads/need-help-with-stopping-bots-spammers.232846/

I could assist people on this site for a fee

 

[Wildcat Media]

Interesting that you should mention that because it's exactly what's happened in my line of work. I've spent my entire working life working within the music industry, as a composer, performer, producer and engineer, and all of those areas are being decimated by AI at an alarming rate.

It's very disturbing. My daughter went to art school and graduated five years ago, just before all of this AI nonsense exploded. She, her art school friends and roommates (many of them still hang out together), and the art community in general, is completely against what AI is doing to their livelihood. Even with her roomies, all three of them in the house work in non-art-related jobs.

Just as an aside--I've been looking into doing stained glass as a hobby. As if I need another one. There are people on Etsy selling stained glass patterns that are not even possible to make. Glass can only be cut and ground in a small number of ways, and these AI-generated patterns being sold are made by LLMs that are not aware of the context of making actual cuts in the glass. AI is taking over other parts of Etsy also--it's a cheap, easy way for these "creators" to make money. And Etsy won't stop them because they make money from the sale.

As for the music industry...Rick Beato on his YouTube channel has really been on top of what's been happening in AI and in music. I've never used Spotify*, never will, but it seems that Spotify and YouTube both let a lot of AI content onto their platforms because...they can monetize it for their own gains. He's even commented about the explosion of AI content on Spotify. Out of curiosity, I tried one of the platforms to make a music intro for one of my own YouTube projects and...yeah, it's listenable as background music but on the standpoint of composition and music theory, you can tell it's both copied from others and disorganized. (My chops are rusty--I can't really make my own music suitable for listening these days. )

It's turning into a class division--AI is funded and promoted by Big Tech with their billions of dollars and multi-billionaire CEOs, while those of us without billions are the ones who are "paying" for this with misleading AI slop content, AI taking away jobs, AI hammering sites run by people like us, AI feeding us incorrect answers when we ask it questions...all to pad the pockets of the stockholders of AI oligarchs.

(* I've been a Qobuz subscriber since they became available in the US; they are the most artist-friendly of all the streamers, and have a manifesto to keep AI-generated content off of their platform.)