[DigitalPoint] App for Cloudflare® 1.9.1.1

[AndrewSimm]

Yep, got that in config.php, and all the API tokens are set up. If I go into the R2 configuration in the Cloudflare add-on, and try to activate one of the two options, I do get a choice of my existing R2 buckets, and everything else in the Cloudflare add-on is appearing and working OK.

Unless I'm missing a step. Although the one media gallery item in the bucket is also displaying properly (and the file is appearing in the bucket), so something is working right.

I just find it interesting that the remaining images in the gallery didn't break after making the change.

And then I start wondering if Cloudflare is caching those older images. The oldest was dated back to mid December of 2022, and it loaded with no issues. But...

Maybe I'll stop being curious and just go ahead and migrate everything over...

If you have moved your data folder over then it is pulling from that folder. I am actually not sure where XFMG uses the internal_data thumbnails. Is your external data url set? Not talking about the config update, that only adds the thumbnails from internal_data and not the ones in data.

 

[Sirmadsen]

Right, I need to do some testing before moving platform all together. Worst case I might have to limit file sizes to 5GB.

Would you say plupload is the better alternative here as it sounds to me appending as it comes in would be less time consuming for the end process?

Seems like chunked upload is the only way to go since Cloudflare have a limit on 100MB per request.
I'm guessing that changing the DNS won't back to my original hosting DNS completely disables Cloudflare including R2?

 

[Wildcat Media]

If you have moved your data folder over then it is pulling from that folder. I am actually not sure where XFMG uses the internal_data thumbnails. Is your external data url set? Not talking about the config update, that only adds the thumbnails from internal_data and not the ones in data.

No, I haven't touched the data directory yet. All I have in any of the R2 buckets is one single file, for the one new media gallery photo. Its thumbnails are still in the data directory on my server.

Not a big deal anymore as I'm deleting the new gallery item, which will leave an empty bucket. Let's hope this works...

 

[Wildcat Media]

Another snag.

The internal_data subdirectories have been migrated and are working properly.

The data directory, though, wasn't so successful. It broke things, so I immediately disabled it. The data directory contents have been migrated. Maybe I'm not understanding what is supposed to happen with my selections here:



The way I'm understanding it...

1) "This subdomain should not already exist." It didn't. Yet I didn't see it added to my domain records, and there was no "Domain access" set up with the bucket either.

2) " Setting a value here will set this XenForo installation to use that URL as the external data URL (you can change or remove the external data URL at any time under External service providers options." Mine was blank--I took this as meaning it would be filled in automatically.

I also noticed this--the domain did not populate my list of R2 buckets in the XF Cloudflare add-on admin:



Maybe it doesn't like a domain like data.forum.mysite.com? I tried it another way (forumdata.mysite.com) and that did not work either.

So tomorrow I guess I'll try manually 1) creating a new DNS record for forumdata.mysite.com and 2) entering that domain in the "External data URL" on the Service Providers page. Cloudflare shows a public URL for the bucket--I could get the IP address from that. Do I need to make that bucket public first so it will work with XF?



I will have to figure out how to set the data R2 bucket up and XF settings for it manually...

Nothing in the XF error logs. And I already had the R2 bucket created, and chose it from the list of available buckets. (I figured I needed to have all the contents moved over before I turned on the R2 bucket.)

 

[digitalpoint]

If you go to Settings under that bucket in Cloudflare, the Connect Domain option is where you can give it a public URL manually.

I’ve seen a couple other Cloudflare accounts that aren’t able to do it automatically via the addon for some reason. But haven’t been able to replicate it or able to see it firsthand to try and debug, but you can do it manually.

 

[Alpha1]

Do you have an actual use case for it besides just having a bigger number for "guests online"?

I'm not concerned about guests online, but its important to see which threads and other content are gaining a lot of views. People tend to post more if their threads are gaining a lot of views and less if there doesnt seem to be a lot of views.

 

[Deleted member 184953]

Cookie consent stuff isn't at the top of my priority list if that's your question? I'm sure it could be worked out, but no plans to as of this second. That's not to say I won't in the future, just that as of this exact moment in time, it's not something I'm working on (or care about).

However, consent to cookies is something that concerns a large part of XF users. In the current state of things would you recommend not to activate guest caching for those who consent to cookies is important because it could produce random behavior that would harm the use of the forum? There is a warning message before activating this function, it's true, but maybe it should be more explicit about the consequences?

 

[digitalpoint]

However, consent to cookies is something that concerns a large part of XF users. In the current state of things would you recommend not to activate guest caching for those who consent to cookies is important because it could produce random behavior that would harm the use of the forum? There is a warning message before activating this function, it's true, but maybe it should be more explicit about the consequences?

Yep, I already added a note about the cookie consent thing for the next version. Don’t think there’s a realistic way around it with the advanced cookie consent option because it renders pages differently based on potentially an infinite number of consent options. Even if you could serve a cached page based on a user having an exact match of consent options, you pretty much lose the advantage of guest page caching at that point because each page has 5 or 10 or even 100 different combos of consent for that many different cached versions. You definitely can’t do it based on the content of the page to see if any of the consent options were applied to hide things, so you’d need to just assume every page is blocking something from every possible consent option.

Pretty sure it’s going to be the same problem with XenForo’s built-in guest caching mechanism.

 

[digitalpoint]

Now if XenForo hid non-consented content with JavaScript instead of server-side stuff, you could probably do guest page caching with the advanced cookie consent system.

Considering it’s beta, maybe a good time for someone to make a feature request to switch it to JavaScript based so it could work with XenForo’s built in guest page caching system.

 

[K a M a L]

Yep, I already added a note about the cookie consent thing for the next version. Don’t think there’s a realistic way around it with the advanced cookie consent option because it renders pages differently based on potentially an infinite number of consent options. Even if you could serve a cached page based on a user having an exact match of consent options, you pretty much lose the advantage of guest page caching at that point because each page has 5 or 10 or even 100 different combos of consent for that many different cached versions. You definitely can’t do it based on the content of the page to see if any of the consent options were applied to hide things, so you’d need to just assume every page is blocking something from every possible consent option.

Pretty sure it’s going to be the same problem with XenForo’s built-in guest caching mechanism.

Xenforo already handeled this on page cache id generator

             $consentedCookieGroups = @json_decode(
                $request->getCookie('consent', '[]'),
                true
            );
            sort($consentedCookieGroups);
            $cookieConsentId = md5(implode(',', $consentedCookieGroups));

            $uri = $request->getFullRequestUri();
            $uri = preg_replace('#(\?|&)_debug=[^&]*#', '', $uri);

            $cacheId = 'page_' . sha1($uri) . '_' . strlen($uri) . "_s{$styleId}_l{$languageId}_c{$cookieConsentId}_v" . self::CACHE_VERSION;

which allows serving different versions of the cached page .

 

[digitalpoint]

Ya, you just can have a zillion different page identifiers with the advanced cookie consent system and no real way to know of anything was hidden on the page due to consent options.

Say you only had 3 options for consent… on/off for each gives you 2 possible options for each. 2^3 (8) possible consent combos. Multiply that by potentially how many different data centers could be caching each possible combo (290 right now). So having 3 possible consent options and all of a sudden you have 2,320 different locations/consent options, which makes guest page cache hits a 1 in 3,320 thing for a user to hit the cache. And that number doubles every additional consent option there is/added.

So yes, it could technically be done, but your cache hit rate starts to get very, very low at that point. At which point using guest page caching becomes rather moot.

Maybe I’ll do it just to do it so people can do it if they really want, but having thousands of different versions for each page really defeats the purpose of guest page caching.

 

[Kirby]

Yeah, XenForo "solves" this by making the consent part of the cache key.
But that pollutes the cache and makes it pretty inefficient - up to the point where it becomes completely uselsss.

The stock consent has 4 states

• No consent al all (cookie is missing) or only consented to to required cookies
• Consent to required and optional
• Consented to required and 3rd party
• Consentd to required, optional and 3rd party

So if a forum has 2 guest-usable styles and 2 guest-usable languages that requires 16 different versions (2 styles * 2 languages * 4 consent states) for every cached page - that's what I'd call pretty inefficient.

Ad it could get even much worse if consent is implemented "properly" by Add-ons (eg. allowing individual consent for every single 3rd party service)

For now the only realistic option would be to disable caching if xf_consent is present, which basically means only first hits can cache content (and receive cached responses).

IMHO it was a bad design decision by XenForo to vary HTML for guests based on consent status - this should not be done.
Every other CMP (at least those I do know) handles this purely in JS on the client.

I pointed out this issue even before 2.2.12 was released, but nobody seemed to care much

Implemented - Update Cookie Banner compliant to GDPR

Hi there I want propose the implementation of the Cookie Banner to be compliant to the GDPR law we have in Europe. Right now the banner have only the possibility to click on "Accept" and this is not accordant with the law we have here about the privacy policy. To be legal here, that banner must...

xenforo.com

Ironically, XenForo does put everything that is needed (at least for embeds) into the HTML and the JS is capable of enabling this after consent is given - it's just not used this way, lol.

 

[digitalpoint]

I’ve been meaning to mention this for awhile but I always forget, but talking about data centers multiplying the number of cached versions of things (applies to everything not just guest page caching), reminded me to say something about Cloudflare’s Tiered Cache. People should enable it in their Cloudflare settings. It’s Cloudflare’s answer to the problem of caches become less and less efficient the more data centers they add.

Tiered Cache works by dividing Cloudflare’s data centers into a hierarchy of lower-tiers and upper-tiers. If content is not cached in lower-tier data centers (generally the ones closest to a visitor), the lower-tier must ask an upper-tier to see if it has the content. If the upper-tier does not have the content, only the upper-tier can ask the origin for content. This practice improves bandwidth efficiency by limiting the number of data centers that can ask the origin for content, which reduces origin load and makes websites more cost-effective to operate.

Tiered Cache · Cloudflare Cache (CDN) docs

Tiered Cache uses the size of Cloudflare’s network to reduce requests to customer origins by dramatically increasing cache hit ratios. With data centers around …

developers.cloudflare.com

 

[Svoboda]

Upgraded to 1.5.1 and everything still works aside from Manage Country Blocking.

Still get the same "Oops! We ran into some problems. Please try again later. More error details may be in the browser console." when I click on that item. Any ideas where to look to get this working? I'm a 99.5% US site and I have a couple countries I'd really love to block as they're racking up requests.

 

[digitalpoint]

Upgraded to 1.5.1 and everything still works aside from Manage Country Blocking.

Still get the same "Oops! We ran into some problems. Please try again later. More error details may be in the browser console." when I click on that item. Any ideas where to look to get this working? I'm a 99.5% US site and I have a couple countries I'd really love to block as they're racking up requests.

Do you have anything in your browser console or XenForo's error log?

 

[digitalpoint]

digitalpoint updated [DigitalPoint] Cloudflare with a new update entry:

Minor update/bug fixes

• More tuning of logic for when to do guest page caching
• Made change so other addons that are also extending the filesystem mount class are able to do so with backward compatibility
• Fixed cosmetic issue with overflow of R2 logs in overlay window
• Prevent users from using the same bucket for public and private areas (prevent users from exposing internal-data as a public bucket)
• Added note about style, language and advanced cookie consent in XF 2.2.12+ to known limitations...

Read the rest of this update entry...

 

[digitalpoint]

Re: advanced cookie consent issue... It's not just the cookie consent thing that would be problematic with exponentially multiplying the number of cached versions stored at the network edge. It's also an issue when a site allows multiple styles or languages to be set on a per user basis. The style/language selected by a registered user isn't stored in a cookie, so there's no way to discern that info from the network edge (before the request goes to the server). So a site that allows multiple styles or languages is also going to have issues with guest page caching.

 

[Kleazy]

Re: advanced cookie consent issue... It's not just the cookie consent thing that would be problematic with exponentially multiplying the number of cached versions stored at the network edge. It's also an issue when a site allows multiple styles or languages to be set on a per user basis. The style/language selected by a registered user isn't stored in a cookie, so there's no way to discern that info from the network edge (before the request goes to the server). So a site that allows multiple styles or languages is also going to have issues with guest page caching.

I save style ID in a cookie for all users and guests so that a software outside of XF can adapt to the style selector.
A XF user on here wrote an add-on for that for me. Works vowel well and was very straight forward to implement.

With Guest-Caching enabled the style selection stays consistent 90% of the time for guests. Never for the forum root. For nested thread views mostly yes.

 

[digitalpoint]

The issue with something like that is there's not any logic you can use at the network edge to make determinations. You basically can just see if a cookie is set, not really see what it's set to and then use caching logic from there. So all you could really so is choose not to serve cached pages if the cookie was set (which then we are back to defeating the whole purpose of guest page caching).

Basically guest page caching at the network edge is never going to be a universal solution for everyone. There's always going to be drawbacks and limitations. If your site doesn't use advanced cookie consent, doesn't have multiple styles/languages and you don't care about view counters incrementing with every view or the number of guests online showing accurately, it could be viable.

 

[Kirby]

The issue with something like that is there's not any logic you can use at the network edge to make determinations. You basically can just see if a cookie is set, not really see what it's set to and then use caching logic from there.

Are you sure about that?
As far as I understand Cloudflare documentation on Cache Keys this should be possible (though only for Enterprise customers).

Other CDNs seem to offer similar functionality:

Introducing CDN Vary Cache By Cookie Value

We are continuing our mission of building the world's most powerful edge caching platform with yet another update to our Vary Cache features. To power some of the most complex caching scenarios we are excited to introduce Vary Cache By Cookie Value. With just a few clicks and a list of cookie...

bunny.net

Another option probably would be to turn xf_language and xf_style into custom request headers on the edge and Vary on them - but that would most likely require to route all non-static requests through a worker (which might be too expensive).
For requests going through the service worker on clients, this should be possible there as well (but probably a bit dificult to do as service workers do not have direct access to cookies (I am looking at you Firefox and Safari for not (yet) supporting CookieStore).

Basically guest page caching at the network edge is never going to be a universal solution for everyone. There's always going to be drawbacks and limitations.

That's the primary reason why I find this topic very interesting

I've experimented with edge caching quite a lot over the last ... 10 years or so and never got it working in a useful way without hitting showstoppers, lol.