[DigitalPoint] App for Cloudflare® 1.10.1

[Ridemonkey]

It already works that way.

Of course it does.

Sorry, was working on a very old and very low-resource XenForo site that I help administer and was looking for ways to reduce server resource consumption, so I was hand-modifying my CF config and that occurred to me. I should have checked my actual site to see the settings before I posted!

 

[Chromaniac]

just curious if it is possible to block countries while allowing main search bots on free accounts. or that feature is locked behind paid accounts? if it is, maybe it can be added to the block country functionality of this addon. and if the official cloudflare bot list is based on IPs and not user-agents which is being spoofed heavily these days.

 

[puterfixer]

It is possible - first rule is to whitelist bots, ie to identify them and the action is to skip all remaining rules. Then from 2nd rule onwards you have conditions on countries or ASNs or whatever with the action to block or challenge the connection.

 

[Chromaniac]

would be nice to be able to control it from the backend. i am very much considering blocking access to usa if i can let googlebot etc through lol. i also noticed that the rule generated by the addon lists individual countries separately which consumes a lot of characters from the allowed set. iirc my manual rule was able to add countries in a single block. would have to check it again.

 

[digitalpoint]

would be nice to be able to control it from the backend. i am very much considering blocking access to usa if i can let googlebot etc through lol. i also noticed that the rule generated by the addon lists individual countries separately which consumes a lot of characters from the allowed set. iirc my manual rule was able to add countries in a single block. would have to check it again.

Ya, I've been meaning to change it to use the new(ish) in{} expression similar to what I changed the cache rule based on extensions to do. Much more efficient, I just haven't had the time to do it yet.

 

[eDaddi]

The simplest solution I can think of would be to have an addon that stores some attachments in the data bucket (ones you want public) and present them as such. Then you don't really need to change anything else or do any kludgey trickery elsewhere. It wouldn't be an impossible addon (and honestly not really that hard), however it's not really a Cloudflare thing, so not sure it would be best suited in this addon.

I took your advice and thought I'd share results for others that are using this add-on, so CF users.... Hope you don't mind me sharing here.

So yea ... Every time an image in a post was displayed (some posts having up to 20 pics, some thread pages having 30 posts ), XF runs a full PHP request for permission check + generates a signed, uncacheable redirect to R2 storage. Uncacheable by the user's browser as well. General traffic and crawlers hitting threads constantly, so each image = multiple PHP hits on every crawl. That saturated our PHP workers.

The small add-on that stores attachments from publicly-viewable forums in our public, edge-cached bucket and serves them as direct URLs. Public images now serve straight from Cloudflare's cache on initial load ... no PHP, no permission check per image view. DMs and private forums are untouched ... they still get the full per-view permission check, so nothing leaks.

The public URL for the image also allowed the user's browser to cache the image as well

Before -> After (3-day windows on the /attachments/ path):

• Presign redirects through PHP: 1,638,000 → 139,000 (−91%)
• Origin timeouts (5xx): 575,000 → 85,000 (−85%)
• Total PHP-touching attachment events: ~2.2M → ~224K

It's been live and stable for weeks now. The drop landed over the first week as edge caches warmed and it's held steady since. The remaining traffic is mostly legitimate private content (DMs, private forums) that's supposed to go through PHP.

Members even state: But the site is now quicker, cleaner, and images are loading faster than I've ever seen here. Must be thirty time!

Net result: the crawler flood that was knocking us over now serves from cache instead of hitting the origin.

To your other point of this not being a part of this add-on .... I see both sides. People use this for CF performance / management .... this certainly helps that but on the other hand ... should probably be a XF codebase fix really.

 

[digitalpoint]

To your other point of this not being a part of this add-on .... I see both sides. People use this for CF performance / management .... this certainly helps that but on the other hand ... should probably be a XF codebase fix really.

Ya, but my main point is that there's nothing in it that necessarily has anything to do with Cloudflare. It would be pretty weird if people needed to install a Cloudflare addon if they wanted that function where they just want to move their private attachment data to public (for example if they didn't use Cloudflare or R2). I was just saying it would be better suited as it's own addon so anyone could use it (even those storing attachments in the local file system like XenForo does by default).

The nice thing about XenForo's abstracted filesystem is people using such an addon would be able to use it regardless of where they store their attachments.

 

[eDaddi]

Totally agree. In my mind aside from proxy'd IPs, is the caching/image optimization ... I assumed it was for most which is why I hinted at it being a helper there. But you said the part about this being a part, or add-on to, XF's filesystem better than I did.

Imagine the cumulative horsepower their hosted forums would save.

 

[Ivancas]

@digitalpoint is there any reason why svgs like https://xenforo.com/community/data/local/icons/regular.svg are not being added/replaced by the R2 bucket?

 

[digitalpoint]

XenForo treats data/local differently. It always keeps them local (hence the name).

I believe it has to do with cross-site issues with how the icons are rendered. The browser would block them if they were on a different hostname.

 

[constants]

I'm helping to set up a brand new forum running on Xenforo Cloud, and we're running into some problems with the R2 setup.

Under ACP > Cloudflare > R2 (object storage), I was able to create the 3 necessary buckets. Logging into Cloudflare, I was able to confirm the buckets were created successfully. I also confirmed that on the ACP > Setup > Options > Cloudflare page, "External Data Url" was filled in correctly for the data bucket endpoint.

However, when I try to upload a new avatar or image attachment, I get broken images. The urls of these broken images are under the external data url endpoint. When I go to ACP > Content > Attachments, I can click on the broken images to successfully view the uploaded image on the Xenforo server (e.g. https://<forumdomain>/admin.php?attachments/<attachmentname>-jpg.9/view), which indicates the upload to the server succeeded. However, when I check the R2 buckets, the files do not exist there. There are no avatars in the data bucket, no attachments in the attachments bucket, and no attachment thumbnails in the xfmg bucket.

I've checked ACP > Logs > Server error log and there are no new logs.

Additionally, going to ACP > Cloudflare > R2 (object storage) and checking the Class A operation logs, I only see ListObjects and ListMultipartUploads actions logged. There is no evidence of a PutObject or UploadPart action ever being performed.

According to the ACP > Setup > Options > Cloudflare page, the API token appears to have all necessary permissions (all permissions are listed in green).

Since we're running on Xenforo Cloud, we do not have server access.

Is there something obvious I'm missing here? Would appreciate any help. Thanks in advance!

 

[digitalpoint]

Anything in the Cloudflare R2 logs (on Cloudflare page in XenForo admin)?

Can you post a screenshot of that page?

 

[constants]

Anything in the Cloudflare R2 logs (on Cloudflare page in XenForo admin)?

Can you post a screenshot of that page?

Do you mean this page? Class A and Class B operations for data.

 

[digitalpoint]

Do you mean this page? Class A and Class B operations for data.

Looks as it should. Only thing I can think of would be if you had a different addon that was doing something to attachments as they are uploaded, but maybe it’s not reading/writing them properly (via abstracted file system).

Not sure what that would be, but do you have any other addons that are doing something with attachments? Like maybe trying to optimize them or something?

 

[constants]

Looks as it should. Only thing I can think of would be if you had a different addon that was doing something to attachments as they are uploaded, but maybe it’s not reading/writing them properly (via abstracted file system).

Not sure what that would be, but do you have any other addons that are doing something with attachments? Like maybe trying to optimize them or something?

These are the only add-ons we have installed. It's not obvious to me which of these might cause a problem.

 

[digitalpoint]

As a test, are attachments able to be uploaded with addon disabled (just in the normal local filesystem)? Internally, attachments are uploaded to the local filesystem before being sent to the R2 bucket. So would be worth checking to confirm attachments work normally.

 

[constants]

As a test, are attachments able to be uploaded with addon disabled (just in the normal local filesystem)? Internally, attachments are uploaded to the local filesystem before being sent to the R2 bucket. So would be worth checking to confirm attachments work normally.

Yes, I have tried disabling R2, and attachments and avatar uploads to the xenforo server both work.

 

[digitalpoint]

I guess you could try making a new API token… if that token is somehow not allowing write permissions to buckets, you’d see something like you are seeing (it seems reads are working, but not writes).

Is your domain valid/in your Cloudflare account?

I just double checked my sites and they all seem to be working normally with R2 (doesn’t seem to be a general issue going on with R2). Unfortunately I don’t have access to a XenForo Cloud instance though, so I can’t really test anything there. But I do know there are a ton of sites using it with R2… so not really sure what could be unique to your site somehow.

 

[constants]

Yes, the domain is valid and in the Cloudflare account.

I'll try making a new API token and report back. Thank you for the suggestion!

 

[constants]

I guess you could try making a new API token… if that token is somehow not allowing write permissions to buckets, you’d see something like you are seeing (it seems reads are working, but not writes).

Is your domain valid/in your Cloudflare account?

I just double checked my sites and they all seem to be working normally with R2 (doesn’t seem to be a general issue going on with R2). Unfortunately I don’t have access to a XenForo Cloud instance though, so I can’t really test anything there. But I do know there are a ton of sites using it with R2… so not really sure what could be unique to your site somehow.

We tried with a new API token, but, unfortunately, it hasn't seemed to make a difference.
Any other ideas of things we could look into?