XF 2.3 Icons in local-data://

[digitalpoint]

I'm curious why there's a special folder within data that is treated differently? I know it's for icons in 2.3, but I'm wondering why someone would want those to be local rather than in whatever abstracted filesystem the site was using for data.

Just trying to figure out if there's something I'm missing and why I wouldn't want to push the icons to R2 (or whatever else)...

 

[Chris D]

I'll ping @Jeremy P to confirm but I believe the primary reason is due to cross-origin issues serving SVG-based spritesheets.

 

[Jeremy P]

Yeah, CORS issues basically.

 

[digitalpoint]

So could probably work around it with properly set CORS policies, but also probably more trouble than it's worth.

 

[digitalpoint]

It might be worth looking at how my Cloudflare addon is doing abstracted filesystem mount points. One of the things is handles (with full backward compatibility) is the ability to do "sub-adapters".

So instead of having adapters for:

• data
• internal-data
• local-data
• code-cache

You could have adapters for:

• data
• internal-data
• data/local
• internal-data/code-cache

It gives you some extra flexibility. Some ways I use it:

• My FileSystem addon allows internal-data/keys to reside in the XenForo data registry without needing to create different static adapters, while leaving the rest of internal-data with the local adapter
• My Cloudflare addon allows someone to have internal-data/attachments stored in R2, but the rest of internal-data stays in the local adapter (normally with XenForo, it's an all or nothing... send everything in internal-data to a specific adapter, or nothing).

Did I mention it maintains backward compatibility (there are a lot of XenForo sites using my Cloudflare addon without any issues... all of them have "sub-adapter" support underneath it all with the abstracted filesystem even if they aren't using R2... without issue).

My version of FsMounts adds a few other thing as well:

    /**
     * This is based on the XenForo standard method, with some extra stuff:
     * - Ability to add adapters based on admin options with static::getDpFsAdapters() method (allows setup of adapters via UI)
     * - Allows disable_asserts to be set on a per-adapter basis
     * - Automatically merging "sub-adapters" from config.php
     * - Returning an extended version of \League\Flysystem\MountManager (sub-adapter support)
     */

Basically when the MountManager handles a request, it looks to see if we have a specific adapter for the sub-directory we are in... if there is, use that... if not, fall back to the adapter for the root path.

As a bonus, if local-data was done as a sub-adapter (data/local) rather than a new full adapter, nothing would have broken between 2.2 and 2.3.

It would also allow you guys to inject your own adapters without the need to edit config.php (R2 support in XenForo Cloud for example) and (probably) wouldn't need you guys to have a special FsMount class for XF Cloud.

 

[Kirby]

It might be worth looking at how my Cloudflare addon is doing abstracted filesystem mount points. One of the things is handles (with full backward compatibility) is the ability to do "sub-adapters".

As already pointed out in the Add-on thread in the past:
Your Add-on is not fully backwards compatible, not even if only mount manager calls are used.

The following code works just fine with standard XenForo, it does not work if your Filesystem Add-on is used with the option to override internal-data://keys

$dir = __DIR__;
require($dir . '/src/XF.php');

XF::start($dir);

$dkimOptions = XF::options()->emailDkim;

if ($dkimOptions['enabled'])
{
        $fs = \XF::fs();

        $keyFile = $dkimOptions['privateKey'];
        $path = 'internal-data://keys/' . $keyFile;


        if ($fs->has($path))
        {
                die("Private Key File Size = " . $fs->getSize($path));
        }
        else
        {
                die('Key File not found');
        }
}
else
{
        die('DKIM is not enabled. Please enable DKIM and test again');
}

 

[digitalpoint]

As already pointed out in the Add-on thread in the past:
Your Add-on is not fully backwards compatible, not even if only mount manager calls are used.

The following code works just fine with standard XenForo, it does not work if your Filesystem Add-on is used with the option to override internal-data://keys

$dir = __DIR__;
require($dir . '/src/XF.php');

XF::start($dir);

$dkimOptions = XF::options()->emailDkim;

if ($dkimOptions['enabled'])
{
        $fs = \XF::fs();

        $keyFile = $dkimOptions['privateKey'];
        $path = 'internal-data://keys/' . $keyFile;


        if ($fs->has($path))
        {
                die("Private Key File Size = " . $fs->getSize($path));
        }
        else
        {
                die('Key File not found');
        }
}
else
{
        die('DKIM is not enabled. Please enable DKIM and test again');
}

Not talking about the FileSystem addon, talking about the underlying way to extend the abstract filesystem.

The Data Registry adapter doesn't return anything with getMetadata() (which is what getSize() uses). Use a different adapter and the above would work just fine having a internal-data://keys/ sub-adapter. The underlying issue isn't that you can't use a sub-adapter in that way, the issue is that you are trying to use a purpose-built adapter (Data Registry adapter) that was intended just for storing keys. And the data registry doesn't store metadata. Using the Data Registry adapter even with the way XenForo does things (as the root of internal-data for example), would yield the exact same issue because that particular adapter doesn't support metadata (which is what you are trying to access with it).

 

[Mouth]

I'd really like/hope to see that all XF files can be abstract/remote, nothing required to be local, to allow for load balancing and cost savings in server hosting.

 

[Jeremy P]

This isn’t really about where the files are stored, it’s about what origin they’re served from. Browsers require SVG <use> elements to be served from the same origin, and they actually lack support for CORS entirely. If you can proxy the requests elsewhere from your web server, you’re free to store the underlying files in object storage or wherever else. The local-data mount is an abstract file system like all the rest.

 

[digitalpoint]

I'd really like/hope to see that all XF files can be abstract/remote, nothing required to be local, to allow for load balancing and cost savings in server hosting.

Serving those files locally is really going to be the simplest option. It's not user generated content that can grow infinitely... it's a finite number of static files (similar to the js folder or the styles folder), and if you are using an edge cache (like Cloudflare) the static content that exists on your web server should be able to be cached just fine.

That being said, as @Jeremy P pointed out, you could move those to a different origin if you really wanted since it's done via XenForo's abstracted file system. For me, I don't particularly see a point of doing that though since it's not user generated content.