[DigitalPoint] App for Cloudflare® 1.8.7

[greenchicken]

bot fight mode plan the next one up from free

 

[greenchicken]

This is whats popping up when I click to turn it on in the admin interface

 

[greenchicken]

as I said it is no problem for me and easily remedied on the CF interface but I have been following your site and tips and tricks for many years and I know you like for all your stuff to be tight

 

[digitalpoint]

Ya... will look into. Might be a little tricky since the setting still is valid, but only to certain zones. So without knowing what plan a zone is on, how do we determine if we should present the setting to users as something they can change or not.

That might need to be something Cloudflare "fixes" on their side. For some settings, Cloudflare properly tags them as changeable or not for the zone (for example Mirage and Polish aren't available to free plans, and Cloudflare marks them as such as we are able to make those options read only/greyed out for plans that don't support it). My gut says Cloudflare should be doing the same thing for the WAF option.

 

[greenchicken]

it's no big deal to me as I said most settings are set and forget anyway. I only need to turn that off when I am adding google adsense code to an ad or something and I don't have to do that often. I appreciate the effort you have put in your stuff over the years. If i not mistaken you used to be on VB and I was too and your forums was what convinced me to switch. Thanks again

 

[digitalpoint]

Honestly, I've had more problems than WAF than I think it's worth (at least if you are using XenForo). WAF is more for situations where a website wasn't particularly built with security in mind. It's looking for HTTP requests that might look malicious because of what's in the payload (something that looks like a SQL query might be a SQL injection, something that looks like it might contain JavaScript might be a JavaScript injection attempt, etc.) XenForo is very solid as far as security goes (things like SQL and JavaScript injections aren't a thing). So at the end of the day, all you are really doing with WAF applied to a XenForo site is messing up your ability to edit templates or post with [code] blocks containing certain things.

 

[Chernabog]

I love what this can do @digitalpoint thank you for providing it to the community. I would really like to add cloudflare to our website/XF community, but I am still a little uncertain how to do so in a way that will ensure everything works as it is supposed to both for the end users and for us in the ACP -- as well as making sure that XF knows the true IP addresses of people so the purpose of site bans and using the stopforumspam functions.

If there is anyone whose very familiar with Cloudflare and uses it on their community installations and would be willing to discuss it with me - that would be awesome!

Appreciate any assistance.

 

[digitalpoint]

There's really nothing special you need to do as far as backend or server config. I'd just setup Cloudflare to be your DNS provider as a first step (even if you don't use Cloudflare's services beyond that, they are in my opinion the fastest and most reliable DNS provider, so...). At that point you can just toggle on/off if you want traffic routed through Cloudflare or direct to your server. You don't need to do anything special as far as getting the true IPs or anything (if your web server isn't doing it automatically, XenForo does it for you automatically without needing to enable or do anything).

It's probably a lot easier than you are thinking in your mind (basically just make them your DNS provider... done.)

 

[Mouth]

Is there an improved way to handle Client Disconnect errors for R2 functionality?



I regularly - eg. daily, sometimes several times a day - get these errors, and it appears that one instance generates 3 error log entries. So when it occurs several times over a few mins, then you get double (or even triple) digit error log entries.

I appreciate the error being logged, but not really sure how to go about resolving it? Appear the add-on has timeout/disconnection errors with the R2 service, but as it occurs almost daily it does seem like either R2 has an ongoing availability issue or the add-on is being quite strict with timeouts or disconnects? And, of course, I wonder what the user experience is when these errors occur? I assume in means that the file/image doesn't load within their browser, as though it was 404.

 

[greenchicken]

Honestly, I've had more problems than WAF than I think it's worth (at least if you are using XenForo). WAF is more for situations where a website wasn't particularly built with security in mind. It's looking for HTTP requests that might look malicious because of what's in the payload (something that looks like a SQL query might be a SQL injection, something that looks like it might contain JavaScript might be a JavaScript injection attempt, etc.) XenForo is very solid as far as security goes (things like SQL and JavaScript injections aren't a thing). So at the end of the day, all you are really doing with WAF applied to a XenForo site is messing up your ability to edit templates or post with [code] blocks containing certain things.

really like the rule to block the register button for the manual persistent spammers.

 

[Mouth]

I regularly - eg. daily, sometimes several times a day - get these errors, and it appears that one instance generates 3 error log entries. So when it occurs several times over a few mins, then you get double (or even triple) digit error log entries.

Eg, all 162 errors in the error log within the last 90mins coming from this add-on ...

 

[digitalpoint]

Ya, still working out a way to better handle 499 errors. The issue isn’t a timeout or anything, rather a general network connection issue on the server. It’s an unexpected disconnect of the client (the client being the server). It’s what would happen if you physically unplugged the server’s Ethernet cable and plugged it back in.

Normally you should only retry 5xx errors as those are issues on the server-side (on Cloudflare’s side). But thinking I might treat 499 as a server side error, which would have it retry once. Can’t really test it though since I’m not getting them ever.

 

[Mouth]

The issue isn’t a timeout or anything, rather a general network connection issue on the server. It’s an unexpected disconnect of the client (the client being the server). It’s what would happen if you physically unplugged the server’s Ethernet cable and plugged it back in.

It doesn't appear that my server or network connection is having any issues, but I'll keep an eye on it ...

:~$ mtr -rwzbc100 6cba001c6e66f6a2962585edfe412c3f.r2.cloudflarestorage.com
Start: 2023-03-24T08:54:18+1100
HOST: <redacted>                                            Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS63949  2600:3c0f:16::5                                0.0%   100    0.5   0.8   0.5   9.7   1.2
  2. AS63949  2600:3c0f:16:35::2                             0.0%   100    0.7   0.6   0.5   0.8   0.1
  3. AS63949  2600:3c0f:16:32::2                             0.0%   100    1.6   2.5   0.9  45.8   5.2
  4. AS63949  2400:8907:100::102                             0.0%   100    0.7   1.1   0.5  20.7   2.3
  5. AS???    13335.syd.equinix.com (2001:de8:6::1:3335:1)   0.0%   100   14.4   5.0   1.1  28.1   6.8
  6. AS13335  2400:cb00:26:3::                               0.0%   100    1.2   5.0   0.8  76.0  10.9
  7. AS13335  2606:4700::6812:95a                            0.0%   100    1.0   1.1   0.8  13.9   1.4

:~$ iostat 1 10
Linux 5.10.0-20-amd64 (<redacted>)     03/24/23        _x86_64_        (1 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          13.35    0.17    2.76    0.10    0.10   83.51

FYI, my hosting (Linode/Akamai) support says;
Taking a look over the errors you provided I wanted to mention the 499 client error I saw in the screenshot. A 499 status code refers to "client closed request" error. This is a client-side code where the client did not wait long enough for the server to respond. It may be worth reviewing the PHP configuration to determine if there is any room for modification to allow for this.

 

[digitalpoint]

There's definitely something "out of place" going on with the 499 errors. They appear to be getting a response back from Cloudflare telling it that it disconnected (if the client disconnected, how did it get the response back telling it that it disconnected before it disconnected?). I posed that question to some Cloudflare engineers, and didn't hear anything back.

I'm starting to wonder if it's a Cloudflare backend error that's being relayed inadvertently. Like I mentioned before, it's kind of hard for me to test because I've never had it happen to me firsthand.

If you do a little digging on the errors you got, are they particularly large attachments? Large enough that the time it takes to transfer them from Cloudflare might be coming into play? It's more going to be network connectivity/bandwidth available to your server rather than the server itself being overloaded.

 

[NealC]

I'm brand new to Cloudflare and thank you for this add-on. I just setup one site, I want to setup a second site. Do I use the same API Token for all sites?

For the proxies it says I need to Pick a cloudflare worker sub-domain. Clicking the link sends me to a 404 on Cloudflare. How do I set this up? I think I figured it out, it's called a Worker service now? I just created the service and deployed/saved. Now your add-on detects it.

On your screenshots you have Firewall rules for bots and spiders. How do I set that up to match yours if that is something recommended?

Same with the User agents for bad actor bot.

I just want to configure this however you suggest. I did read the link below:

Cloudflare optimizations for XenForo

I've had a few people that use my Cloudflare add-on ask me what Cloudflare options/settings are best for XenForo. So rather than repeating myself privately, I'm going to turn this post into a bit of a configuration help document for those that want to read it. Cloudflare is pretty great out of...

xenforo.com

 

[digitalpoint]

I'm brand new to Cloudflare and thank you for this add-on. I just setup one site, I want to setup a second site. Do I use the same API Token for all sites?

You can if you'd like as long as you don't restrict the API token to a single zone/site. For example, I have a single XenForo API token that I use across all my sites.

For the proxies it says I need to Pick a cloudflare worker sub-domain. Clicking the link sends me to a 404 on Cloudflare. How do I set this up? I think I figured it out, it's called a Worker service now? I just created the service and deployed/saved. Now your add-on detects it.

Indeed... it looks like they slightly changed the URL for setting a Worker subdomain. The URL has been updated for the next version.

On your screenshots you have Firewall rules for bots and spiders. How do I set that up to match yours if that is something recommended?

Those rules are specific to my site... hitting specific URLs on my site continuously over and over to try to generate referrer spam. So those aren't really going to work on other sites unless you have the same problem. And even then, the URLs they are hitting are going to be different (since it's a different site).

Same with the User agents for bad actor bot.

Those were just made up for the purpose of the screenshot. Looks more interesting when the rules aren't empty. hah I don't have any user agent blocking actually in production myself.

 

[Chernabog]

There's really nothing special you need to do as far as backend or server config. I'd just setup Cloudflare to be your DNS provider as a first step (even if you don't use Cloudflare's services beyond that, they are in my opinion the fastest and most reliable DNS provider, so...). At that point you can just toggle on/off if you want traffic routed through Cloudflare or direct to your server. You don't need to do anything special as far as getting the true IPs or anything (if your web server isn't doing it automatically, XenForo does it for you automatically without needing to enable or do anything).

It's probably a lot easier than you are thinking in your mind (basically just make them your DNS provider... done.)

Thank you..yeah I was reading through all the posts and all the options and it was going way over my head.. a little overwhelming - so I am sure I've been over thinking the entire idea of using cloudflare. I've just heard they are great against denial of service attacks and general hacking, so really wanted to give our community that level of protection... I will take your advice and see how it goes. Appreciate the feedback.

 

[digitalpoint]

Thank you..yeah I was reading through all the posts and all the options and it was going way over my head.. a little overwhelming - so I am sure I've been over thinking the entire idea of using cloudflare. I've just heard they are great against denial of service attacks and general hacking, so really wanted to give our community that level of protection... I will take your advice and see how it goes. Appreciate the feedback.

My advice is just to start slow... there's no reason you need to "enable all the things" all at once. You'll just make your head spin. Just get them setup as your DNS provider, and go from there.

 

[digitalpoint]

@Mouth so I did some testing and digging and talking to a Cloudflare engineer about how the http client (your server in your case) can get a response back from Cloudflare telling it that it disconnected when it in fact didn't (it got the response after all, so how is it "disconnected").

if the server responds with a 499, it's because the client has disconnected. how that client is disconnecting is another thing.

to my knowledge, yes it is possible when you have multiple hops involved.

in a client a <> client b <> origin sort of setup

client a may still be connected to client b however client b is no longer connected to the origin

generally happens when the timeouts are not correct overlapping and you have a short timeout somewhere in the chain but the others are longer and still connecting.

The client timeout on our side (the addon) is unlimited (there actually is no timeout, so if it takes 100 seconds, it takes 100 seconds... we aren't disconnecting).

What we are left with is stuff that is outside my control (or your control)... if the network connections are being routed through a peer or other network that is dropping/resetting the connection. So for example, say a data center a few hops away that is routing your traffic rebooted the router/switch that your traffic was going through. That might cause it (along with a ton of other things... network congestion somewhere along the route for example).

So I think I'm going to treat 499 as a 5xx (and do a single retry behind the scenes and hop it worked the second try before failing to the error log). Another option would be to suppress 499 errors from the error log, but not sure I like that idea... like if my data center is having networking issues, it's probably a good thing to know.

 

[NealC]

I'm setting up the R2 config and see that you have a bucket for XFMG but what about the XFRM?

Confirm after R2 is setup with your add-on I do not need to change anything in the config.php for attachments to get saved in R2 vs. local? I see posts on the site about S3 and Digital Ocean needing some config in the config.php and want to make sure I've done all setup needed using your add-on.

I copied all of the data to R2 and when I click on an image attachment in a post I'm getting the following error:

The request cannot be loaded. Please try again later.

The URL of the attachment (link address) is: https://data.domain.com/attachments/0/127-7bc42cdad818854bcb54abd70521af41.jpg

I can see the file is in the data bucket in R2.

I did a test post and attached an image and that worked. What am I missing?

I used rclone and set the config per this document. I set acl to private, not sure if that's correct. (Edit: I deleted everything and re-uploaded with acl to public-read and same problem)

rclone · Cloudflare R2 docs

You must generate an Access Key before getting started. All examples will utilize access_key_id and access_key_secret variables which represent the …

developers.cloudflare.com

Edit:

For the image error, if I paste in the image URL it does show. I think the issue is likely that I don't have the correct folder structure uploaded to the R2 buckets. If someone could tell me what the data bucket should look like, i.e. I uploaded my data/* into it so it has that structure. For the attachments bucket I initially uploaded just /internal_data/attachments/* but I think the folder structure is incorrect. Is the bucket supposed to be /internal_data/attachments/*?

FIXED:

That was it, didn't have the proper rooting per this post. It helps to uncheck "view prefixes as directories" on upper right of directory/file list in R2