Crazy amount of guests

[BrettC]

BTW, here's a little interesting tidbit from yesterday's traffic surge.

How many desktop users actually use Linux as an OS? Very small number in comparison to Windows and everything else. Yet...

View attachment 344636

Given the demographics of the forums I admin (they tend to skew older and less technically-inclined), the number of Linux users would be far, far less than most typical forums out there. (And I don't know if the different Linux builds like Ubuntu, Mint, etc. advertise themselves as "Linux" or "Ubuntu," so...there's also that.) CF may also be lumping those into Unknown/Others also (which on CF's part, doesn't help me at all).

The SmartTV is also interesting. At first I would think it might be the compromised TVs but, since they are residential proxies that are most likely spoofing user agents, these might actually be legitimate smart TVs accessing us. (Which seems clumsy.)

Do what you will with that information. For my part, if they show Linux in http.user_agent, they get a challenge. (Don't panic--that is after my whitelisting.)

That's an interesting find there. While it is true that linux uptake has been increasing over the years... it isn't anything like that!

On a Debian 13 install with Firefox, this is the user agent that I see: Mozilla/5.0 (X11; Linux x86_64; rv:153.0) Gecko/20100101 Firefox/153.0 - very basic, nothing special. X11 makes it fairly clear it's using the X window system, and the Linux bit just reaffirms it's, well... Linux!

 

[Alvin63]

Well I think I can stop looking at the guest list for a while. I had to block some more countries as got fed up with looking up ASN's and the residential proxies just get seen as real browsers and bypass managed and interactive challenges. But the numbers are between 250 and 800 now in the guest list at various times.

So I've used all 5 rules now

1) Block on China, Russian Federation, Myanmar and Belarus - no google presence here so haven't whitelisted known bots. Am keeping this one permanently.
2) Country block, whitelisting known bots, on Bangladesh, Brazil, Egypt, Indonesia, Kazakhstan, Mexico, Philippines, Thailand, Vietnam
3) ASN Blocks - many of these but got fed up with doing that hence rule 2 for the worst offenders
4) URI path interactive challenge for whats new, search and members - about 20% of any who haven't been blocked still get through.
5) Country interactive challenge for 64 countries - all of which have been in the guest list at some point - this catches any who hit somewhere else on the forum rather than the 3 paths in rule 4.

I don't like completely blocking some countries but hopefully can unblock them at some point.

Also still have bot fight mode on which is catching zillions - mainly from big data centres in the USA - AWS, Microsoft, Alibaba and more.

 

[Wildcat Media]

That's an interesting find there. While it is true that linux uptake has been increasing over the years... it isn't anything like that!

On a Debian 13 install with Firefox, this is the user agent that I see: Mozilla/5.0 (X11; Linux x86_64; rv:153.0) Gecko/20100101 Firefox/153.0 - very basic, nothing special. X11 makes it fairly clear it's using the X window system, and the Linux bit just reaffirms it's, well... Linux!

I haven't had time to fire up one of my Ubuntu desktop systems recently, but I get the impression it may also say "Linux" rather than Ubuntu.

Just an hour later after that traffic spike I posted, look at where Linux sits, more where I'd expect it to. And I suspect a good portion of my whiltelisted bots are categorized as Unknown/Others during this time, with others possibly using Linux.



So that huge inrush of Linux user agents lined up with our spike in traffic...by an insane amount. I agree, it's an interesting data point...

 

[Alvin63]

Also just wanted to say a huge thanks to @badabugs1 for all the great help setting up the .htaccess cloudflare secret key. It does work fine - I don't seem to have any bypassing cloudflare and hitting the origin IP, which is good.

 

[Anthony Parsons]

I don't like completely blocking some countries but hopefully can unblock them at some point.

You don't need to block countries or ASN's or IP's. You cannot beat residential proxies that way, because if you have US open, then that is 150 million homes with potential IP's for use to hit you.

Cover your Search, that is load bearing.

(starts_with(http.request.uri.path, "/whats-new") or starts_with(http.request.uri.path, "/search"))
and not cf.client.bot
and not http.cookie contains "xf_user="

The next one is covering fake browsers, this is many residential proxies and bad bots.

(((http.user_agent contains "Chrome/1" and not any(lower(http.request.headers.names[*])[*] == "sec-ch-ua"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "sec-fetch-mode"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "accept-language")))) and (not cf.client.bot) and (not http.cookie contains "xf_user")

This is the real heart of residential proxies...

(http.user_agent contains "Chrome/120." or http.user_agent contains "Chrome/133." or http.user_agent contains "Chrome/146." or http.user_agent contains "Chrome/145." or http.user_agent contains "Firefox/135." or http.user_agent contains "Firefox/133." or http.user_agent contains "Chrome/131." or http.user_agent contains "Chrome/116." or http.user_agent contains "Chrome/124." or http.user_agent contains "Chrome/107." or http.user_agent contains "Chrome/104." or http.user_agent contains "Chrome/110." or http.user_agent contains "Chrome/136." or http.user_agent contains "Chrome/99." or http.user_agent contains "Chrome/119." or http.user_agent contains "Chrome/123." or http.user_agent contains "Chrome/100." or http.user_agent contains "Chrome/101." or http.user_agent contains "Chrome/142." or http.user_agent contains "Chrome/65." or http.user_agent contains "Firefox/147." or http.user_agent contains "Firefox/144." or http.user_agent contains "Chrome/108." or http.user_agent contains "Chrome/112." or http.user_agent contains "Chrome/109." or http.user_agent contains "Chrome/117." or http.user_agent contains "Chrome/103." or http.user_agent contains "Chrome/106.") and http.referer eq "" and (not cf.client.bot) and (not http.cookie contains "xf_user")

Set all of them to non-interactive challenge. You DO NOT need to block when these are going to block the automatic and allow the human from all countries. Notice how each one allows cf known bots and xf users logged in.

I would basically place them in reverse order to how I have listed them, search last, that way anything automated gets caught in the first two. Not a country blocked, ASN, IP, na da. You don't need to if you are trying to block the bad stuff but allow everything else.

If you want to then block a known good bot, you now have two rules remaining where you can do that if you wish, as some known bots are aggressive and not so good. Meta is at the top of that list.

I only run two of those, I have the browser impersonators first, search second. Between the two, for me, that takes out the majority of the nasties, BUT, I have a server that can handle the other noise.

 

[Alvin63]

You don't need to block countries or ASN's or IP's. You cannot beat residential proxies that way, because if you have US open, then that is 150 million homes with potential IP's for use to hit you.

Cover your Search, that is load bearing.

(starts_with(http.request.uri.path, "/whats-new") or starts_with(http.request.uri.path, "/search"))
and not cf.client.bot
and not http.cookie contains "xf_user="

The next one is covering fake browsers, this is many residential proxies and bad bots.

(((http.user_agent contains "Chrome/1" and not any(lower(http.request.headers.names[*])[*] == "sec-ch-ua"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "sec-fetch-mode"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "accept-language")))) and (not cf.client.bot) and (not http.cookie contains "xf_user")

This is the real heart of residential proxies...

(http.user_agent contains "Chrome/120." or http.user_agent contains "Chrome/133." or http.user_agent contains "Chrome/146." or http.user_agent contains "Chrome/145." or http.user_agent contains "Firefox/135." or http.user_agent contains "Firefox/133." or http.user_agent contains "Chrome/131." or http.user_agent contains "Chrome/116." or http.user_agent contains "Chrome/124." or http.user_agent contains "Chrome/107." or http.user_agent contains "Chrome/104." or http.user_agent contains "Chrome/110." or http.user_agent contains "Chrome/136." or http.user_agent contains "Chrome/99." or http.user_agent contains "Chrome/119." or http.user_agent contains "Chrome/123." or http.user_agent contains "Chrome/100." or http.user_agent contains "Chrome/101." or http.user_agent contains "Chrome/142." or http.user_agent contains "Chrome/65." or http.user_agent contains "Firefox/147." or http.user_agent contains "Firefox/144." or http.user_agent contains "Chrome/108." or http.user_agent contains "Chrome/112." or http.user_agent contains "Chrome/109." or http.user_agent contains "Chrome/117." or http.user_agent contains "Chrome/103." or http.user_agent contains "Chrome/106.") and http.referer eq "" and (not cf.client.bot) and (not http.cookie contains "xf_user")

Set all of them to non-interactive challenge. You DO NOT need to block when these are going to block the automatic and allow the human from all countries. Notice how each one allows cf known bots and xf users logged in.

I would basically place them in reverse order to how I have listed them, search last, that way anything automated gets caught in the first two. Not a country blocked, ASN, IP, na da. You don't need to if you are trying to block the bad stuff but allow everything else.

If you want to then block a known good bot, you now have two rules remaining where you can do that if you wish, as some known bots are aggressive and not so good. Meta is at the top of that list.

I only run two of those, I have the browser impersonators first, search second. Between the two, for me, that takes out the majority of the nasties, BUT, I have a server that can handle the other noise.

Thank you. Would those browsers not block genuine browsers as well? I do have the uri search and whats new path with not xf_user and not cf.client bot. I think probably 99.9% of them are only going direct to whats new.

Also I'm not sure what non interactive challenge is or why that rather than interactive challenge?

 

[Alvin63]

You don't need to block countries or ASN's or IP's. You cannot beat residential proxies that way, because if you have US open, then that is 150 million homes with potential IP's for use to hit you.

Cover your Search, that is load bearing.

(starts_with(http.request.uri.path, "/whats-new") or starts_with(http.request.uri.path, "/search"))
and not cf.client.bot
and not http.cookie contains "xf_user="

The next one is covering fake browsers, this is many residential proxies and bad bots.

(((http.user_agent contains "Chrome/1" and not any(lower(http.request.headers.names[*])[*] == "sec-ch-ua"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "sec-fetch-mode"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "accept-language")))) and (not cf.client.bot) and (not http.cookie contains "xf_user")

This is the real heart of residential proxies...

(http.user_agent contains "Chrome/120." or http.user_agent contains "Chrome/133." or http.user_agent contains "Chrome/146." or http.user_agent contains "Chrome/145." or http.user_agent contains "Firefox/135." or http.user_agent contains "Firefox/133." or http.user_agent contains "Chrome/131." or http.user_agent contains "Chrome/116." or http.user_agent contains "Chrome/124." or http.user_agent contains "Chrome/107." or http.user_agent contains "Chrome/104." or http.user_agent contains "Chrome/110." or http.user_agent contains "Chrome/136." or http.user_agent contains "Chrome/99." or http.user_agent contains "Chrome/119." or http.user_agent contains "Chrome/123." or http.user_agent contains "Chrome/100." or http.user_agent contains "Chrome/101." or http.user_agent contains "Chrome/142." or http.user_agent contains "Chrome/65." or http.user_agent contains "Firefox/147." or http.user_agent contains "Firefox/144." or http.user_agent contains "Chrome/108." or http.user_agent contains "Chrome/112." or http.user_agent contains "Chrome/109." or http.user_agent contains "Chrome/117." or http.user_agent contains "Chrome/103." or http.user_agent contains "Chrome/106.") and http.referer eq "" and (not cf.client.bot) and (not http.cookie contains "xf_user")

Set all of them to non-interactive challenge. You DO NOT need to block when these are going to block the automatic and allow the human from all countries. Notice how each one allows cf known bots and xf users logged in.

I would basically place them in reverse order to how I have listed them, search last, that way anything automated gets caught in the first two. Not a country blocked, ASN, IP, na da. You don't need to if you are trying to block the bad stuff but allow everything else.

If you want to then block a known good bot, you now have two rules remaining where you can do that if you wish, as some known bots are aggressive and not so good. Meta is at the top of that list.

I only run two of those, I have the browser impersonators first, search second. Between the two, for me, that takes out the majority of the nasties, BUT, I have a server that can handle the other noise.

Ok Claude had analysed all the server logs previously so I asked it to check which Chrome versions my bot swarm was using and it was Chrome/148, 149, and 150 across all six suspect user-agents, so maybe rule 3 wouldn't work for these bots and I couldn't add those versions or it would block genuine users presumably? Those six user agents are behind the worldwide residential proxies apparently. They are very sneaky bots.

Your rule 2 is really helpful thank you.

These are the numbers

 

[puterfixer]

With the free CF account limited to 5 rules, there would be room for optimization, depending on the action being taken and the right logical order. For example:

1. Top defense for the forum: if user is not logged in and tries to hit the URLs that are reserved for registered users (manage my account, private messages, what's new), then the action should be to entirely BLOCK the access regardless of any other condition. These requests make no sense for a guest and are costly for the server. Maybe consider adding viewing members' profiles for privacy, or forum searching to prevent excessive load on the server, and then replicate the same principles in the forum's permissions for Guest group - even if a guest manages to get through CF, the forum should also prevent them from completing that kind of request.
2. Exception for bots, if you want to: if visitor is validated by CF as being a bot, then SKIP all remaining rules, so that you don't have to keep adding this condition as exception in the following rules. This however relies on CF properly identifying bots based on the combination of source IP + user-agent + whatever else CF does; it's not enough to trust a visitor saying they're a Google spider but they are connecting from Tencent in China - that's clearly fake.
3. Sources you know for sure are scraper bots, and want to target them explicitly: if user-agent matches those keywords, then CHALLENGE them (pick one of the options, see which is more effective). With this separate rule, 1) you make their actions more costly before blocking them, and 2) you get separate statistics on their number and success rate.
4. Sources you want to completely block: if visitor's IP is from a list of ASNs (datacenters), OR visitor's country is from a list of country codes (where you have no regular visitors from), OR visitor's user-agent matches some keywords (bad bots), then BLOCK them.
5. Same as previous, but for sources you want to challenge, in case they're a mix of legitimate guests and bots; same kind of filters, but the action is one of the challenges.

 

[smallwheels]

You don't need to block countries or ASN's or IP's. You cannot beat residential proxies that way, because if you have US open, then that is 150 million homes with potential IP's for use to hit you.

Once more: The question is what you want to achieve. If you want to block scrapers ASN blocks can be largely beneficial - not all scrapers come via client isp networks. Tecent, AWS, M247 Europe and countless others can simply and cleanly be blocked and this is per definition way more effective and more reliable than any challenge or other rule for those. Some countries are absolutely worth blocking as well if you don't have regular audience from there, given the bot waves via residential proxies coming from them. I.e. China, Brazil, Argentina and Vietnam.

It makes absolutely no sense to leave your pants completely down just b/c the US are such a spamhole that one may not be able to block as a country b/c of having regular users coming from there. But even within the US there are countless data center ASNs that are absolutely worth blocking completely which won't generate any harm at all.

If you only focus on residential proxies via sip client networks you leave out a larger bunch of the problem and the bunch that is way easier to get rid of.

 

[Levina]

You don't need to block countries or ASN's or IP's. You cannot beat residential proxies that way, because if you have US open, then that is 150 million homes with potential IP's for use to hit you.

Yep. Found that out myself when I started with blocking just about every country not in the western world. It worked for a short while and then the residential proxies came in from countries I can't block, most of all from the US. Turned off all country blocking rule as well as the handful of ASN numbers I had in a second custom rule. Switched to monitoring what they were targeting: a whole lot of paths none were allowed, including some nonsensical ones for WordPress. Blocked those and put a managed challenge on paths like /members/, /misc/cookies, /search/ and /whats-new/. I also have a custom rule that blocks specific bots that aren't welcome.

My visitor count has been stable between 150-250 for a couple of months now. I know there are still some residential proxies among them, but they have no access to what I blocked and as long as they are viewing threads or media items and don't start hammering new paths where they are not welcome, I'm fine with it. You will never get all of them.

I also think the order of the custom rules matter. Looking into that now because I may have that set wrong.

 

[Anthony Parsons]

Would those browsers not block genuine browsers as well?

Non-interactive means its automatic for real browsers, it stops everything else. This is how not to piss off your legitimate users. Don't serve user a challenge, it just upsets people. Non interactive recognises real browsers, real people behind them, and lets them straight through. Everything else is trying to fake being a browser, and that is detectable. The rules I provided you are robust and if you set it non interactive, you will have zero issues from real users. No countries, asn or ip blocks, are needed. The main residential proxy rule I gave above, that will target older browsers, which is what most proxies use. Most people have updated browsers, BUT, for those still using old OS and browser versions, are not interrupted or disturbed by having to tick a challenge or such. You DO NOT need to block or manage challenge rules unless you are doing something very specific and intended, non interactive is the choice you want to not upset any real people.

 

[Alvin63]

With the free CF account limited to 5 rules, there would be room for optimization, depending on the action being taken and the right logical order. For example:

1. Top defense for the forum: if user is not logged in and tries to hit the URLs that are reserved for registered users (manage my account, private messages, what's new), then the action should be to entirely BLOCK the access regardless of any other condition. These requests make no sense for a guest and are costly for the server. Maybe consider adding viewing members' profiles for privacy, or forum searching to prevent excessive load on the server, and then replicate the same principles in the forum's permissions for Guest group - even if a guest manages to get through CF, the forum should also prevent them from completing that kind of request.
2. Exception for bots, if you want to: if visitor is validated by CF as being a bot, then SKIP all remaining rules, so that you don't have to keep adding this condition as exception in the following rules. This however relies on CF properly identifying bots based on the combination of source IP + user-agent + whatever else CF does; it's not enough to trust a visitor saying they're a Google spider but they are connecting from Tencent in China - that's clearly fake.
3. Sources you know for sure are scraper bots, and want to target them explicitly: if user-agent matches those keywords, then CHALLENGE them (pick one of the options, see which is more effective). With this separate rule, 1) you make their actions more costly before blocking them, and 2) you get separate statistics on their number and success rate.
4. Sources you want to completely block: if visitor's IP is from a list of ASNs (datacenters), OR visitor's country is from a list of country codes (where you have no regular visitors from), OR visitor's user-agent matches some keywords (bad bots), then BLOCK them.
5. Same as previous, but for sources you want to challenge, in case they're a mix of legitimate guests and bots; same kind of filters, but the action is one of the challenges.

Thanks - yes it would be possible to combine some rules and have a free one. I could combine 1 and 2 - 2 was only just added recently - it's just near the top because it was a block rule and I had the luxury of a spare rule, but I could combine those two to free one up. They both block countries but one whitelisted known bots and the other didn't. With the worst offending countries where there is no google presennce I didn't aven add not cf.client.bot - so there was no way it could be faked!

The second country block rule does have not .cf.client.bot it has about 7 countries in - also worst offenders. Things have stabilised since I did that. Third rule is ASN blocks.

So all the blocks at the start and then the next rule (first one after the blocks) is for those three paths - whats new, search and members. It was a managed challenge originally but 20% of them were solving it. So I tried switching to interactive challenge - no difference. That rule also whitelists xf_user and known bots, but it also has a referrer in the rule - so genuine guests don't get challenged - because they would have a forum cookie just for landing on a page. Now I did wonder if this might be letting more bots through if they faked a referer, but the logs show none of them have - they are simply solving the challenge. 20% of them. They also look like genuine up to date chrome browsers.

So it has been clear from the logs what I'm dealing with - countries all over the world with a heavy presence from Eastern, African and Caribbean countries. Eastern mainly

But also they have a single target - that is all - they go straight to whats new - nowhere else. A few try index.php/whats new. If there are any that hit any other path on the forum they get caught in the last rule.

So I focused on trying to avoid disruption for members - avoiding any blocks or challenges for USA Uk and Europe. Using challenges for most of the rest of the world, and blocking the few really tough ones. As well as blocking ASN's for particular countries. Of course that ends up being a bit endless hence blocking those extra 7 countries.

I am ok with not doing anything with the US because it seems botfightmode is handling that side of it and all the ones from data centres it seems. Those do show older browsers or things like python in the header. The ones getting through do look like normal user browsers using a current level of Chrome.

I just decided to a) try and keep the site as normal as possible for members b) try and get the numbers down to an acceptable level without challenging everyone.

99% of them are scraper bots. Other bots were already dealt with for some time in robots.txt and .ht access. Then this almost a million of them hit the site! It looks like there are about 600,000 now.

Anyway if I free up one rule I can add the one from @Anthony Parsons that might catch some that are solving the challenge then I can look at unblocking 7 of the countries. However they are countries I wouldn't usually have any members from - it's possible but highly unlikely and at some point I might be able to unblock those again.

Because all the server logs were analysed, it helped to understand what was going on. Although admittedly Claude isn't 100% reliable but it was useful.

 

[Alvin63]

Non-interactive means its automatic for real browsers, it stops everything else. This is how not to piss off your legitimate users. Don't serve user a challenge, it just upsets people. Non interactive recognises real browsers, real people behind them, and lets them straight through. Everything else is trying to fake being a browser, and that is detectable. The rules I provided you are robust and if you set it non interactive, you will have zero issues from real users. No countries, asn or ip blocks, are needed. The main residential proxy rule I gave above, that will target older browsers, which is what most proxies use. Most people have updated browsers, BUT, for those still using old OS and browser versions, are not interrupted or disturbed by having to tick a challenge or such. You DO NOT need to block or manage challenge rules unless you are doing something very specific and intended, non interactive is the choice you want to not upset any real people.

Thank you - I guess I used the referrer so it didn't affect real users but maybe non interactive is better. None of my users or genuine guests get challenged due to the referrer and whitelisting xf_user although it is possible that a genuine guest might just type the forum address into a browser and then not have a referring cookie so non interactive does sound better. Only one member has mentioned having a challenge and that was just on a work computer but not their own device, so non interactive might be better in that sense too. However anyone browsing from one of the 64 countries in rule 5 would indeed get challenged, so yes non interactive sounds better.

But as 20% of this bot swarm can solve any challenge I had to look at reducing the numbers before they got to the site (which hopefully your rule will help with as well).

Incidentally, how would I set up your rule 2? ie what field? http? header? I've been using the field boxes to set it up. If I just paste the expression, do the field boxes sort themselves out? Also what order would I put it in the rule order please?

After the block rules and before the uri path/whats new challenge rule?

Edit - I just pasted your expression and saved.

 

[Alvin63]

Ok so I have combined rules 1 and 2 country blocking to free up a rule, and added @Anthony Parsons rule (thank you) after blocking rules and before my uri path challenge rules.

It's early days but it is only catching 147 bots out of nearly 600,000 of them at the moment. The issue with my bot swarm I think is they are using real chrome - as Claude analysed the data and showed that - they are actually using real full chome - they just know how to solve the challenges. But maybe it will change over time.

Edit - it's gone up to 726 - so that helps So maybe there are different bot operators involved as well as the 6 main user agents using real chrome. The data centre bots are very different as well. And I guess the number would be higher if I didn't have the country block rule above it.

However since blocking the extra 7 countries yesterday the solve rates for rules 4 and 5 have gone right down. So I've made a note that those 7 countries seem more likely to have the ability to solve challenges than the others that are being challenged. I think.

 

[smallwheels]

Non-interactive means its automatic for real browsers, it stops everything else.

That's what the advertisement says and probably a bit oversimplified. Users don't have to click, CF tries via JavaScript, fingerprinting and other magic to differentiate humans, using real browsers from bots and fake browsers. Which may or may not work. Or may work perfectly today but not tomorrow or not with some fancy new scraper. It is clearly less annoying for users than the interactive challenge and it may be working better, equally good or worse and this can change any time.

 

[Alvin63]

I thought block rules had to come first? But maybe set ups vary. ie they get blocked before they hit the site. If they hit the site they get challenged,.

 

[Alvin63]

I see the point though, if they are weeded out before the block then less need to block. Still trying to get my head round that! At this stage I'd rather keep the blocks for now until the CSR rate comes down for rule 4 - whats new URI path challenge.

However the new rule from @Anthony Parsons is now working well and taken out 1.9k of them now.

 

[Levina]

non interactive is the choice you want to not upset any real people.

Not sure about this, Anthony. I have extensions in my browsers to protect me from trackers and to stop background scripts. That sets a flag in Cloudflare, so when I visit my forum as a not-logged in visitor and click on What's New, I get the interstitial screen. I check the box and I am in.

I use a managed challenge for paths where genuine visitors are welcome, like /whats-new/. But if I were to set a non-interactive challenge genuine visitors with normal browsers that have protection against trackers on board might get a hard block. With a managed challenge genuine visitors at least get a second chance so to speak by confirming they are human.

EDIT: I also thought of false positives. They might get through with a non-interactive challenge but most likely be stopped in a managed challenge.

 

[Alvin63]

Not sure about this, Anthony. I have extensions in my browsers to protect me from trackers and to stop background scripts. That sets a flag in Cloudflare, so when I visit my forum as a not-logged in visitor and click on What's New, I get the interstitial screen. I check the box and I am in.

I use a managed challenge for paths where genuine visitors are welcome, like /whats-new/. But if I were to set a non-interactive challenge genuine visitors with normal browsers that have protection against trackers on board might get a hard block. With a managed challenge genuine visitors at least get a second chance so to speak by confirming they are human.

Well I could change it to a managed challenge and add a referer like I did for the URI path challenge as that stops genuine guests being challenged.

 

[Alvin63]

Ok I've changed it to managed challenge and added a referer. So genuine guests will get a cookie on arriving on the site meaning they have a referer if clicking anywhere else so should be bypassed.

(((http.user_agent contains "Chrome/1" and not any(lower(http.request.headers.names[*])[*] == "sec-ch-ua"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "sec-fetch-mode"))) or (((http.user_agent contains "Mozilla/5.0" and (http.user_agent contains "Chrome/1" or http.user_agent contains "Firefox/1")) and not any(lower(http.request.headers.names[*])[*] == "accept-language")))) and (not cf.client.bot) and (not http.cookie contains "xf_user") and http.referer eq ""