[DigitalPoint] App for Cloudflare® 1.8.8

[Mouth]

Am I understanding this right?

That sounds right, that's been discussed here. I ensure that I don't use any addons that don't use XF native functionality correctly, or bypass it completely, to ensure I don't have such issues.

 

[Fullmental]

That sounds right, that's been discussed here. I ensure that I don't use any addons that don't use XF native functionality correctly, or bypass it completely, to ensure I don't have such issues.

Got it. Unfortunately a lot of these addons are user requests or feature ports from our old platform. It's not easy to rip away features from everyone just because it doesn't play nice the first go at this migration effort.

I'll have to see what we can do. @digitalpoint , is it possible to have only the avatars and attachment public/internal data go to R2? That's really our biggest issue right now, we're out of space on our production drive and buying more is very cost ineffective. We were hoping to offload user attachments, images, and video assets to R2 for this purpose, but we don't really care if other stuff in the /data/ folder stays local. For the time being I'm probably going to go ahead with just internal_data attachments to at least clear up some space, but there's plenty in the /data/avatars and /data/attachments directories as well.

 

[Mouth]

is it possible to have only the avatars and attachment public/internal data go to R2?

Should be what you're looking for ...

R

Post in thread '[DigitalPoint] App for Cloudflare®'

Jan 29, 2023

although if you really want to store those files in R2, you can via config.php edit.

I guess the code will be?

$config['fsAdapters']['internal-data/image_cache'] = function() {
    return \DigitalPoint\Cloudflare\League\Flysystem\Adapter\R2::getAdapter('internal-data/image_cache', 'cf-bucket-name');
};

• rdn

and

Post in thread '[DigitalPoint] App for Cloudflare®'

Dec 12, 2023

XF has the option to redirect the whole data directory to an external source provider.
I want to redirect the video folder to R2 only.

Redirects are going to be setup in your web server, or if you want to do it upstream of your web server, you can setup redirects for your Cloudflare zone under Rules -> Redirect Rules.

If it's just an issue of R2 handling just a single sub-directory (for things like incoming uploads), you can do that via config.php. This is off the top of my head, (I haven't tested it), but this should work:

$config['fsAdapters']['data/video'] = function()
{...

• digitalpoint

 

[Fullmental]

Should be what you're looking for ...

R

Post in thread '[DigitalPoint] App for Cloudflare®'

Jan 29, 2023

although if you really want to store those files in R2, you can via config.php edit.

I guess the code will be?

$config['fsAdapters']['internal-data/image_cache'] = function() {
    return \DigitalPoint\Cloudflare\League\Flysystem\Adapter\R2::getAdapter('internal-data/image_cache', 'cf-bucket-name');
};

• rdn

and

Post in thread '[DigitalPoint] App for Cloudflare®'

Dec 12, 2023

XF has the option to redirect the whole data directory to an external source provider.
I want to redirect the video folder to R2 only.

Redirects are going to be setup in your web server, or if you want to do it upstream of your web server, you can setup redirects for your Cloudflare zone under Rules -> Redirect Rules.

If it's just an issue of R2 handling just a single sub-directory (for things like incoming uploads), you can do that via config.php. This is off the top of my head, (I haven't tested it), but this should work:

$config['fsAdapters']['data/video'] = function()
{...

• digitalpoint

That appears to be the opposite of what I want to do. I want to just exclude anything that doesn't play nice with the existing configuration. Is there a way to do that in the php config as well? My assumption is the cloudflare addon would need adjustment to exclude folders.

Edit: I suppose I could go with the obvious solution that eluded me for several hours just now: keep the impacted files on the local storage and only delete what's known to work right with R2

I will say I noted it's a lot more than just addon resources that were impacted. A lot of the default xf settings were using local relative paths for things like smiles, reactions, even styles & templates. There's a lot of stuff that'll just link to the relative path on the server and that stuff all breaks the moment you remove those files and put them on R2, even if they were in the /data folder. Seems like a lot that just has to be pointed to the right data point, because if you upload a new copy or paste the R2 url it then redirects fine, it's only existing relative paths that break. I wonder if it there is some sort of rebuild job that can fix this automatically, or if it's something that the migration tool in the addon would have done for us, if it didn't time out and cancel every time.

 

[digitalpoint]

That appears to be the opposite of what I want to do. I want to just exclude anything that doesn't play nice with the existing configuration. Is there a way to do that in the php config as well? My assumption is the cloudflare addon would need adjustment to exclude folders.

Edit: I suppose I could go with the obvious solution that eluded me for several hours just now: keep the impacted files on the local storage and only delete what's known to work right with R2

I will say I noted it's a lot more than just addon resources that were impacted. A lot of the default xf settings were using local relative paths for things like smiles, reactions, even styles & templates. There's a lot of stuff that'll just link to the relative path on the server and that stuff all breaks the moment you remove those files and put them on R2, even if they were in the /data folder. Seems like a lot that just has to be pointed to the right data point, because if you upload a new copy or paste the R2 url it then redirects fine, it's only existing relative paths that break. I wonder if it there is some sort of rebuild job that can fix this automatically, or if it's something that the migration tool in the addon would have done for us, if it didn't time out and cancel every time.

You can use the same general methodology. You can either specify just avatars and attachment folders to be on R2, or if you want all of the data folder to be R2, you can specify that and override the folders you don’t want to be in R2.

Underneath it all, you either have data default to local or R2 and then override the sub folders that you want to be the opposite of the default.

 

[John917]

@digitalpoint Do you have any idea if Cloudflare is the reason why I am having a 403 issue when accessing only /account/upgrades page? I have no issue on other pages. I tried disabling the cloudflare addon and tried pausing cloudflare too but no luck.

 

[digitalpoint]

@digitalpoint Do you have any idea if Cloudflare is the reason why I am having a 403 issue when accessing only /account/upgrades page? I have no issue on other pages. I tried disabling the cloudflare addon and tried pausing cloudflare too but no luck.

Doesn’t sound like it if you disabled it and it’s still a problem.

 

[Fullmental]

Hi,

We've been reviewing this addon and we are concerned about the visibility of our cloudflare service configurations to anyone with ACP access. While we trust our staff in general, there are settings in the cloudflare app that we would rather restrict to only those that already have a login to our own cloudflare dashboard for security purposes.

Any chance we can get granular permissions set up so we can control visibility to each of the cloudflare panels?

 

[digitalpoint]

There’s a “Cloudflare” admin permission already, but beyond that (wanting to give some admins access to only certain parts of the Cloudflare config), I don’t see it becoming that granular to be honest.

Just out of curiosity, what’s the scenario where you want to give someone partial access to Cloudflare config? Personally, I don’t think anyone but super admins should have access to any of it.

 

[Fullmental]

There’s a “Cloudflare” admin permission already, but beyond that (wanting to give some admins access to only certain parts of the Cloudflare config), I don’t see it becoming that granular to be honest.

Just out of curiosity, what’s the scenario where you want to give someone partial access to Cloudflare config? Personally, I don’t think anyone but super admins should have access to any of it.

We just found it. We were expecting permissions under the user group/user permissions in the sidebar, but apparently it's under "administrator permissions"

A few things we'd like more users to have access to include the Under Attack and Developer modes, as these are quick, simple toggles to fix temporary issues until an admin can get online to further diagnose.

 

[John917]

Doesn’t sound like it if you disabled it and it’s still a problem.

Ended up being a bug that was fixed in the recent xenforo update!

 

[digitalpoint]

We just found it. We were expecting permissions under the user group/user permissions in the sidebar, but apparently it's under "administrator permissions"

A few things we'd like more users to have access to include the Under Attack and Developer modes, as these are quick, simple toggles to fix temporary issues until an admin can get online to further diagnose.

Internally, the system doesn’t work in a way where individual settings are controlled on their own (all the settings on the Settings page are read in parallel and then compared to what the user has then set to and then the changes are written out). So even if you had a permission for a subset of settings, it’s a bit more complicated with how you would actually implement that (you really would need a completely different mechanism internally that would be quite a bit slower, but more granular).

So at a minimum, you would need to trade-off the ability to set settings fairly quickly in bulk with a much slower way of doing backend API calls for each individual setting. And honestly just not seeing that sort of trade-off worth it for the vast majority of sites/use cases. That being said, it wouldn’t be too hard for you to use the API class to do it with whatever logic you see fit for your site.

 

[Fullmental]

Internally, the system doesn’t work in a way where individual settings are controlled on their own (all the settings on the Settings page are read in parallel and then compared to what the user has then set to and then the changes are written out). So even if you had a permission for a subset of settings, it’s a bit more complicated with how you would actually implement that (you really would need a completely different mechanism internally that would be quite a bit slower, but more granular).

So at a minimum, you would need to trade-off the ability to set settings fairly quickly in bulk with a much slower way of doing backend API calls for each individual setting. And honestly just not seeing that sort of trade-off worth it for the vast majority of sites/use cases. That being said, it wouldn’t be too hard for you to use the API class to do it with whatever logic you see fit for your site.

I guess that makes sense, although I don't really see why that couldn't be handled with just visibility options, sort of like whether or not a given option in the ACP, or edit permissions are given granularly, but I suppose in theory the API key has the same permissions regardless.

One other question. Does this addon allow the integration of the image proxy/image cache directories as well? While the attachment data is great for stuff that is specifically uploaded to our site, we have about 10 years of old images that have been proxied we'd like to offload to R2 as well.

 

[digitalpoint]

I guess that makes sense, although I don't really see why that couldn't be handled with just visibility options, sort of like whether or not a given option in the ACP, or edit permissions are given granularly, but I suppose in theory the API key has the same permissions regardless.

Ya, it’s certainly not impossible. It just wouldn’t be a quick and easy thing. Would need a lot of internal code refactoring, and while not impossible, it does come down to the amount of coding effort needed to make it work and how many users that will benefit. So working on that (which I don’t think would be used by a ton of sites) instead of other things that would be more universally beneficial, probably isn’t a great choice for resource allocation.

One other question. Does this addon allow the integration of the image proxy/image cache directories as well? While the attachment data is great for stuff that is specifically uploaded to our site, we have about 10 years of old images that have been proxied we'd like to offload to R2 as well.

As long as whatever it is uses. XenForo’s abstracted filesystem and exists within the data or internal_data folder, it would work.

 

[Fullmental]

Ya, it’s certainly not impossible. It just wouldn’t be a quick and easy thing. Would need a lot of internal code refactoring, and while not impossible, it does come down to the amount of coding effort needed to make it work and how many users that will benefit. So working on that (which I don’t think would be used by a ton of sites) instead of other things that would be more universally beneficial, probably isn’t a great choice for resource allocation.



As long as whatever it is uses. XenForo’s abstracted filesystem and exists within the data or internal_data folder, it would work.

OK. So how would you add a custom directory in here? I don't see the option anywhere, but we're on a test site right now so maybe it's disabled by some other config change?:

 

[digitalpoint]

If you mean a custom sub-directory within internal_data, see this post:

R

Post in thread '[DigitalPoint] App for Cloudflare®'

Jan 29, 2023

although if you really want to store those files in R2, you can via config.php edit.

I guess the code will be?

$config['fsAdapters']['internal-data/image_cache'] = function() {
    return \DigitalPoint\Cloudflare\League\Flysystem\Adapter\R2::getAdapter('internal-data/image_cache', 'cf-bucket-name');
};

• rdn

 

[Fullmental]

If you mean a custom sub-directory within internal_data, see this post:

R

Post in thread '[DigitalPoint] App for Cloudflare®'

Jan 29, 2023

although if you really want to store those files in R2, you can via config.php edit.

I guess the code will be?

$config['fsAdapters']['internal-data/image_cache'] = function() {
    return \DigitalPoint\Cloudflare\League\Flysystem\Adapter\R2::getAdapter('internal-data/image_cache', 'cf-bucket-name');
};

• rdn

Looks good. I see you don't recommend it due to constant writes, but I imagine that won't be as much of an issue if we keep the cache lifetime at unlimited. Then it'd basically be the same as fetching the images as attachments, if I understand correctly? Also since the entire site is basically served via cloudflare already I kind of doubt we'll see a notable performance change...if anything it'll be better because our server runs so lean to begin with, but we'll test it. Thanks!

 

[digitalpoint]

Looks good. I see you don't recommend it due to constant writes, but I imagine that won't be as much of an issue if we keep the cache lifetime at unlimited. Then it'd basically be the same as fetching the images as attachments, if I understand correctly? Also since the entire site is basically served via cloudflare already I kind of doubt we'll see a notable performance change...if anything it'll be better because our server runs so lean to begin with, but we'll test it. Thanks!

Yep… not something I would do, but I’m not the boss of you.

 

[bbqchef33]

i hate asking questions that were probably asked before, but i searched for 20 minutes and couldn't find the answer. So, apology in advance if already addressed.

Ive recently migrated from a VB forum on dedicated servers to xenforo cloud. I've downloaded this addon and installed it on my xenforo cloud hosted forum. Got everything set up and opened a ticket to have them run the php cmd.php db:migrate-data. They responded that this cannot be done on the cloud hosting and needs to be run from a browser. Im trying to migrate all my attachments, avatars, gallery, etc to R2. All thats left is to get everything over to R2. Is this supported on cloud hosted sites.? If so, what do i do next?

 

[digitalpoint]

i hate asking questions that were probably asked before, but i searched for 20 minutes and couldn't find the answer. So, apology in advance if already addressed.

Ive recently migrated from a VB forum on dedicated servers to xenforo cloud. I've downloaded this addon and installed it on my xenforo cloud hosted forum. Got everything set up and opened a ticket to have them run the php cmd.php db:migrate-data. They responded that this cannot be done on the cloud hosting and needs to be run from a browser. Im trying to migrate all my attachments, avatars, gallery, etc to R2. All thats left is to get everything over to R2. Is this supported on cloud hosted sites.? If so, what do i do next?

The migration tool is only done via CLI... so if your hosting doesn't support CLI, then there really isn't another way to do it.