• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Flysystem adapter implementations for cloud storage

Sim

Well-known member
#1
It would be useful to provide a couple of flysystem adapter implmeentation for popular cloud storage providers (eg AWS S3 / Rackspace) so that people can confidently use them out of the box without relying on 3rd party addons (regardless of how simple those addons may be to implement).

It's quite important to have something we can use which is fully tested and supported - especially for such a fundamental feature which has the potential to break large parts of our site if it goes wrong.
 

Sim

Well-known member
#3

Kirby

Well-known member
#4
I haven't actually tested it, but I don't think you'll need an Add-on, just a config:
PHP:
$config['fsAdapters'] = [
    'data' => function () {
        return new \League\Flysystem\Filesystem(
            new \League\Flysystem\AwsS3v3\AwsS3Adapter(
                new \Aws\S3\S3Client([
                    'credentials' => [
                        'key'    => 'your-key',
                        'secret' => 'your-secret'
                    ],
                    'region' => 'your-region',
                    'version' => 'latest|version',
                ]),
                'your-bucket-name'
        ));
    },
];
and the adapter itself including it's dependencies.
 

Sim

Well-known member
#5
I don't think you'll need an Add-on ... [just] the adapter itself including it's dependencies.
How do you propose to get the adapter itself into a location where the XF core can include it? :p

I recognise it's likely going to be a fairly trivial addon - but it's still an addon and my point remains - it would be good if several could be provided either as part of the core, or by way of supported addon.
 

Mike

XenForo developer
Staff member
#6
The main thing would be loading these files, though you could likely put them virtually anywhere and include the necessary autoloader. The AWS SDK is a huge amount of overhead (in terms of size/files to include in the core download) for very few people who might potentially use it, for example. (We have done this with a couple libraries already but we could theoretically load a huge amount more to cover all sorts of edge cases.)
 

Chris D

XenForo developer
Staff member
#7
I haven't actually tested it, but I don't think you'll need an Add-on, just a config:
PHP:
$config['fsAdapters'] = [
    'data' => function () {
        return new \League\Flysystem\Filesystem(
            new \League\Flysystem\AwsS3v3\AwsS3Adapter(
                new \Aws\S3\S3Client([
                    'credentials' => [
                        'key'    => 'your-key',
                        'secret' => 'your-secret'
                    ],
                    'region' => 'your-region',
                    'version' => 'latest|version',
                ]),
                'your-bucket-name'
        ));
    },
];
and the adapter itself including it's dependencies.
Slightly more complete example (you need to override the data path too to ensure paths are prefixed with the bucket URL):
PHP:
$config['fsAdapters'] = [
   'data' => function()
   {
      $client = new \Aws\S3\S3Client([
         'credentials' => [
            'key'    => 'ABC',
            'secret' => '123'
         ],
         'region' => 'eu-west-2',
         'version' => 'latest',
      ]);

      return new \League\Flysystem\AwsS3v3\AwsS3Adapter($client, 'a.a.com', 'data');
   }
];

$config['externalDataUrl'] = function($externalPath, $canonical)
{
   return 'https://s3.eu-west-2.amazonaws.com/a.a.com/data/' . $externalPath;
};
 

Kirby

Well-known member
#8
How do you propose to get the adapter itself into a location where the XF core can include it? :p
Put it wherever you want and call it's autoloader :)
If this is what you mean with "requires an XF 2 Add-on" you're right.
But as Mike has already pointed out, it'd dont't think its a viable option to bundle a huge amount of dependencies that (this is my assumption) the majority of customers would never need.
 

Chris D

XenForo developer
Staff member
#9
It's not an invalid suggestion. For example we could roll our own AWS client and Flysystem adapter which is much more lightweight and only covers the very specific endpoints we need to handle S3.

Bear in mind the AWS client that the official Flysystem S3 adapter uses covers every aspect of AWS which is humongous. It'd be nice actually if they componentised it a bit. Compressed it's 1.25MB. Uncompressed it's nearly 8MB.

But, as I said in the other thread and has been said here, advanced users can most likely roll it themselves, or an add-on might be able to simplify and roll the process up slightly (even if it's just a DIY tutorial) so something like that should be sufficient in the meantime.
 

Jake B.

Well-known member
#10
Slightly more complete example (you need to override the data path too to ensure paths are prefixed with the bucket URL):
PHP:
$config['fsAdapters'] = [
   'data' => function()
   {
      $client = new \Aws\S3\S3Client([
         'credentials' => [
            'key'    => 'ABC',
            'secret' => '123'
         ],
         'region' => 'eu-west-2',
         'version' => 'latest',
      ]);

      return new \League\Flysystem\AwsS3v3\AwsS3Adapter($client, 'a.a.com', 'data');
   }
];

$config['externalDataUrl'] = function($externalPath, $canonical)
{
   return 'https://s3.eu-west-2.amazonaws.com/a.a.com/data/' . $externalPath;
};
I assume something similar should work for internal_data as well?