Crazy amount of guests

We have about a 50/50 mix of desktop/mobile and around 1m pageviews/month, so it'd be a substantial amount of people to challenge and would probably affect us negatively I think :(
It could be written to minimise the number of real people challenged though.

1) Whitelist member cookie so logged in members don't get challenged.
2) Referer equals "" - most real guests will have a referer from a search engine or a home page before clicking on a thread so won't get challenged. However that will let a proportion of bots through as some fake a referer. And some real guests or logged out members don't have a referer if coming from Apple Spotlight or via a bookmark eg.
3) Exclude your main country from the rule - unless it's the country where most of the bots are coming from!

Anyway no easy answer.
 
It could be written to minimise the number of real people challenged though.

1) Whitelist member cookie so logged in members don't get challenged.
2) Referer equals "" - most real guests will have a referer from a search engine or a home page before clicking on a thread so won't get challenged. However that will let a proportion of bots through as some fake a referer. And some real guests or logged out members don't have a referer if coming from Apple Spotlight or via a bookmark eg.
3) Exclude your main country from the rule - unless it's the country where most of the bots are coming from!

Anyway no easy answer.
The USA is half of our traffic as well as the most valuable one, and the bots are largely coming from there right now. It's tough to find a balance. I'll consider giving it a try though!
 
The USA is half of our traffic as well as the most valuable one, and the bots are largely coming from there right now. It's tough to find a balance. I'll consider giving it a try though!
Yes that is the problem - most are coming from the USA for me too. Or maybe just challenge Chrome 154 and Chrome 150. See if they solve it. If they can't then any real guests using those browsers would just get a challenge rather than a block.
 
Yes that is the problem - most are coming from the USA for me too. Or maybe just challenge Chrome 154 and Chrome 150. See if they solve it. If they can't then any real guests using those browsers would just get a challenge rather than a block.
Oh I've tried challenges, they solve it at about a 20-25% rate. I'm constantly seeing them successfully flood our old, irrelevant threads, some from multiple decades ago. The block was a last resort. 😅 I wish I could let real visitors though but it just hasn't worked out and I've been trying to mitigate it for weeks. The fact that I've not gotten any emails about getting blocked despite it being nearly 6 hours since implementing it says something. Will see how it goes.
 
Oh I've tried challenges, they solve it at about a 20-25% rate. I'm constantly seeing them successfully flood our old, irrelevant threads, some from multiple decades ago. The block was a last resort. 😅 I wish I could let real visitors though but it just hasn't worked out and I've been trying to mitigate it for weeks. The fact that I've not gotten any emails about getting blocked despite it being nearly 6 hours since implementing it says something. Will see how it goes.
Ah sorry :-)
 
Oh I've tried challenges, they solve it at about a 20-25% rate. I'm constantly seeing them successfully flood our old, irrelevant threads, some from multiple decades ago. The block was a last resort. 😅 I wish I could let real visitors though but it just hasn't worked out and I've been trying to mitigate it for weeks. The fact that I've not gotten any emails about getting blocked despite it being nearly 6 hours since implementing it says something. Will see how it goes.
I don't know how your server infrastructure is setup, but have you considered implementing a Proof of Work Challenge on a limited basis?
 
We have about a 50/50 mix of desktop/mobile and around 1m pageviews/month, so it'd be a substantial amount of people to challenge and would probably affect us negatively I think :(
Same here, and one forum skews more towards desktops. It's not a good metric to base filters on.

I did find that challenging all Linux accesses helps, but allowing those that are logged in. I only allow a small handful of CF-identified bots, and block all the others, so the important ones are not affected.

But the bots have altered over the past month and more of them seem to identify as Windows.
 
I'm constantly seeing them successfully flood our old, irrelevant threads, some from multiple decades ago.
Same here. The two oldest have threads dating back to 2002-2003.

This makes me wonder if there is a way we could limit access to older threads, based on referrer or injected header from CF (by way of signaling if the bot is part of cf.verified.bots ), but have some sort of gateway for human visitors to actually view them. We could keep current threads in the active forums, but move the automatically locked threads into archive forums (as a sub-forum) after a set amount of time. Then we would have our own challenge to identify humans, something unique the bots wouldn't figure out (or we can easily change if they did). We don't want to limit archives to members only, in other words.

I don't know our ratio of open to locked threads, but when you are managing a forum approaching 39 million posts and a million threads, reducing that footprint starts to make more sense.

As a side note, I wonder which CF challenges (interactive vs. non-interactive) bots are solving more easily, or if it's about equal.
 
Back
Top Bottom