[DigitalPoint] FileSystem 1.0.1

[digitalpoint]

digitalpoint submitted a new resource:

[DigitalPoint] FileSystem - Local Cluster and Data Registry filesystem adapters

Allows you to optionally use a Local Cluster filesystem adapter for code-cache:// (allows you to keep file system changes in sync with different web servers... things like template/phrase changes as well as add-on development). Could also be used simply to trigger any command after file writes (maybe check changes into a repository automatically).

Allows you to store the files in internal-data://keys/ in XenForo's data registry.

Both functions can be enabled...

Read more about this resource...

 

[digitalpoint]

I should also point out that while I use this add-on internally for my sites (specifically the part that keeps multiple servers in sync), the reason I made a Data Registry adapter and released it was so other add-on developers could see how it's done (management of adapters without config.php edits as well as sending just part of an abstracted filesystem to different adapters. For example, someone could make an easier to manage S3 adapter or a MySQL adapter where files are stored in a database (like maybe just sitemaps only or something. Just random ideas here...

 

[Kirby]

Unfortunately this does not seem to work properly for me

The following code works fine with standard XenForo:

\XF\Util\File::writeToAbstractedPath('internal-data://keys/test.txt', 'Just some test');

$fs = \XF::fs()->getFilesystem('internal-data');
$fileContents = $fs->read('/keys/test.txt');

echo($fileContents . "\n");

\XF\Util\File::deleteFromAbstractedPath('internal-data://keys/test.txt');

and outputs Just some test

With the keys override active I do get the following error:

An exception occurred: [League\Flysystem\FileNotFoundException] File not found at path: keys/test.txt in src/vendor/league/flysystem/src/Filesystem.php on line 390
#0 src/vendor/league/flysystem/src/Filesystem.php(179): League\Flysystem\Filesystem->assertPresent('keys/test.txt')
#1 [internal function]: League\Flysystem\Filesystem->read('keys/test.txt', Array)
#2 src/vendor/league/flysystem-eventable-filesystem/src/EventableFilesystem.php(431): call_user_func_array('League\\Flysyste...', Array)
#3 src/vendor/league/flysystem-eventable-filesystem/src/EventableFilesystem.php(395): League\Flysystem\EventableFilesystem\EventableFilesystem->callFilesystemMethod('read', Array)
#4 src/vendor/league/flysystem-eventable-filesystem/src/EventableFilesystem.php(141): League\Flysystem\EventableFilesystem\EventableFilesystem->delegateMethodCall('read', Array)
#5 test.php(10): League\Flysystem\EventableFilesystem\EventableFilesystem->read('/keys/test.txt')
#6 {main}

Can this be fixed so the above code does work correct if the keys override is active as you've explained in https://xenforo.com/community/threads/granular-filesystem-adaptor.210906/post-1603600 that this would work?

 

[digitalpoint]

Do you have the addon installed? From the looks of that backtrace, it doesn’t even look like it’s installed to me.

 

[Kirby]

Of course the Add-on is installed and active and the keys override is enabled

Here is the full reproduction code (place it as dp-fs-bugtest.php in XF root and execute it via CLI):

$dir = __DIR__;
require($dir . '/src/XF.php');

XF::start($dir);

\XF\Util\File::writeToAbstractedPath('internal-data://keys/test.txt', 'Just some test');

$fs = \XF::fs()->getFilesystem('internal-data');
$fileContents = $fs->read('/keys/test.txt');

echo($fileContents . "\n");

\XF\Util\File::deleteFromAbstractedPath('internal-data://keys/test.txt');

 

[digitalpoint]

Oh, I see what's going on... you are first calling for a specific filesystem, and then taking that file system and reading from it. Rather than making the request in one go. If you get the filesystem "internal-data", it's going to restrict itself to that exact filesystem.

So the write was working, but not the read. If you change this:

$fs = \XF::fs()->getFilesystem('internal-data');
$fileContents = $fs->read('/keys/test.txt');

...to this:

$fileContents = \XF::fs()->read('internal-data://keys/test.txt');

It works as expected.

Basically it's doing some internal trickery with the first directory of the path to see if there's a secondary filesystem available, but since you called explicitly for the exact file system "internal-data" and it doesn't know about the path at that point, you get what you ask for.

 

[digitalpoint]

Long story short is you need to let the MountManager class (\XF::fs()) do it's magic rather than try to be smarter than it and decide what filesystem you care about. If you look in the XF\Mail\Mail class where the DKIM key is read, there's no requesting an explicit filesystem going on there.

$path = 'internal-data://keys/' . $dkimOptions['privateKey'];
$keyFile = \XF::fs()->read($path);

 

[Kirby]

Oh, I see what's going on... you are first calling for a specific filesystem, and then taking that file system and reading from it. Rather than making the request in one go. If you get the filesystem "internal-data", it's going to restrict itself to that exact filesystem.

[...]

Basically it's doing some internal trickery with the first directory of the path to see if there's a secondary filesystem available, but since you called explicitly for the exact file system "internal-data" and it doesn't know about the path at that point, you get what you ask for.

Yes, I know how Flysystem and the Mount-Manager does work, I've written adapters myself .

Thats' why I specifically asked about such code in the development discussion thread and you insisted your approach would not break existing code while I argumented that I don't see any way to achieve this without implementing a union filesystem.

So the question remains:
Can you fix / change your code without implementing a union filesystem so the above example does work fine?

 

[digitalpoint]

If you are talking about being able to do it while bypassing the MountManager class (which XenForo never does), probably not. I mean maybe (haven't looked into it)... but I write for stuff to work with XenForo and how XenForo works. If someone wants to get an absolute specific filesystem and bypassing the MountManager XenForo uses, I guess they could also make sure they are calling for the right filesystem too.

This would probably work (I didn't test it):

$fs = \XF::fs()->getFilesystem('internal-data/keys');
$fileContents = $fs->read('/keys/test.txt');

But then you are getting away from the whole point of having an abstracted filesystem (and mount manager which is deciding where to route requests to).

If you grep through XenForo, there's only one instance of getFilesystem string, and it's from a different class. There's never a case in XenForo where they try to be smarter than the MountManager class.

Again, you could if you really wanted to, but you need to make sure you call the right filesystem (it's not "internal-data" in the case of the keys directory). But I can't really think of a use case where you ever would want to do that (you are always going to be better off using the normal \XF::fs()->read() call). What is your use case where you feel it's better to bypass the MountManager class XenForo uses?

If you look at the methods you are using in XF\Util\File, you are calling the normal MountManager there for write() and delete(). Why are you wanting to not call the MountManager class for read()?

 

[digitalpoint]

digitalpoint updated [DigitalPoint] FileSystem with a new update entry:

Avoid method naming collision

Minor update to avoid method naming collision with XFCloud.

The dependency on App for Cloudflare® has been raised to version 1.8.1 for the same reason (you will need to upgrade this as well as the App for Cloudflare® addon).

Read the rest of this update entry...