To make sure that internal data stays ... internal
Overview - Hetzner Docs
docs.hetzner.com
Yeah I know - hopefully they'll have more realistic limits for production use
Using DigitalOcean Spaces or Amazon S3 for file storage
- Thread starter Chris D
- Start date
docs.hetzner.com
ostrerov
Member
Ok, maybe I should use different buckets, although I'm sorry, I still don't understand why, if the endpoint itself is protected by login and password, but never mind. How do I then specify different URLs forTo make sure that internal data stays ... internal
Overview - Hetzner Docs
Yeah I know - hopefully they'll have more realistic limits for production use
data and internal_data?expanserpb
Active member
Hmm was fine on 2.2 but now getting a lot of open_basedir problems because its trying to traverse outside of restrictions:
- ErrorException: Template error: [E_WARNING] is_readable(): open_basedir restriction in effect. File(/nonexistent/.aws/config) is not within the allowed path(s): (/var/www/)
- src/vendor/aws/aws-sdk-php/src/DefaultsMode/ConfigurationProvider.php:152
XenForo updated Using DigitalOcean Spaces or Amazon S3 for file storage in XF 2.1 & XF 2.2 with a new update entry:
Improved compatibility with XenForo 2.2
Read the rest of this update entry...
Improved compatibility with XenForo 2.2
Read the rest of this update entry...
@PapaTango this should hopefully help.
expanserpb
Active member
Out of interest, if I ping up a test instance of my live build (from backups) and it was connected to the space object storage (as it would be on restoration) what would happen?
Let's say I uploaded a new avatar on test host whilst the real host was live?
Let's say I uploaded a new avatar on test host whilst the real host was live?
ostrerov
Member
It is advisable to use separate storages or folders for test applications.
If the test has all the settings identical, and the paths too, then the production application will also be replaced
That would be a bad idea.
You would be affecting the file system that your live site is running on. Any changes you make on your test install would affect your live install.
expanserpb
Active member
So:
- Put site into maintenance mode
- Backup flat files and sql
- Copy object storage to a new space or different folders
- Restore to test instance
- Change config php to point at restored object storage
- Take test site out of maintenance mode and do test work (e.g. Major upgrades)
SatGuyScott
Active member
Just got a notice saying that the resource was updated.
It say the add in is no longer required for XF 2.3
But then it says the update is to benefit customers still running XF 2.3
I think you meant 2.2 there.
It say the add in is no longer required for XF 2.3
But then it says the update is to benefit customers still running XF 2.3
I think you meant 2.2 there.
SatGuyScott
Active member
Nope didn't see the change, I must have been typing the other post at the same time.
Thanks!
PapaTango
Member
If I were still a devout, a candle would be lit for you!
I will give this
v2.4.0 update a spin around the PHP tree later tonight.Hopefully, I won't get hit in the head by any falling S3 configuration rocks...
Outcome report to follow.
beerForo
Well-known member
So disable it before the upgrade, and delete after? Anything else we need to know or configure?
PapaTango
Member
I have so many new knots on my head I cannot even begin to count them...
This:
It all started with a WHM backup. Oh, and just because I exported the raw flat files and did a Maria export to my local workstation...
The platform baseline is running PHP v8.1 fully provisioned. XF v2.2.16, XFAws v2.3.0. It all works perfectly. The first step was to upgrade the XFAws addon to v2.4.0--via drag and drop. It did not like that a bit and choked with a bunch of red error message lines. Something having to do with the location of flags for a Snog addon. I disabled that and another potential troublemaker, and uploaded again. The upgrade went smoothly until I went into the site.
No posted images. All gone. Interestingly, S3 transport worked fine with the avatar, thumb, and internal data config directives. Time to see if something changes with v8.2. No, and now my admin panel has thrown me out and I cannot get past the login window. Time to reload the backup and restart Apache.
I tried several iterations of this, including installing the upgrade from the
Reload.
So, I am back to running the baseline profile and consulting with Dr. Kush for my hurting head.
The PHP directives are as follows:
This:
[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/S3/RegionalEndpoint/ConfigurationProvider.php on line 83[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/DefaultsMode/ConfigurationProvider.php on line 85[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/Endpoint/UseFipsEndpoint/ConfigurationProvider.php on line 82[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/Endpoint/UseDualstackEndpoint/ConfigurationProvider.php on line 83[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/EndpointDiscovery/ConfigurationProvider.php on line 86[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/Retry/ConfigurationProvider.php on line 88[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/ClientSideMonitoring/ConfigurationProvider.php on line 90[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/S3/UseArnRegion/ConfigurationProvider.php on line 83[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/S3/RegionalEndpoint/ConfigurationProvider.php on line 83[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/DefaultsMode/ConfigurationProvider.php on line 85[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/Endpoint/UseFipsEndpoint/ConfigurationProvider.php on line 82[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/Endpoint/UseDualstackEndpoint/ConfigurationProvider.php on line 83[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/EndpointDiscovery/ConfigurationProvider.php on line 86[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/Retry/ConfigurationProvider.php on line 88[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/ClientSideMonitoring/ConfigurationProvider.php on line 90[12-Oct-2024 20:15:25 UTC] PHP Deprecated: Use of "self" in callables is deprecated in /home/papata5/fotofora/src/addons/XFAws/_vendor/aws/aws-sdk-php/src/S3/UseArnRegion/ConfigurationProvider.php on line 83It all started with a WHM backup. Oh, and just because I exported the raw flat files and did a Maria export to my local workstation...
The platform baseline is running PHP v8.1 fully provisioned. XF v2.2.16, XFAws v2.3.0. It all works perfectly. The first step was to upgrade the XFAws addon to v2.4.0--via drag and drop. It did not like that a bit and choked with a bunch of red error message lines. Something having to do with the location of flags for a Snog addon. I disabled that and another potential troublemaker, and uploaded again. The upgrade went smoothly until I went into the site.
No posted images. All gone. Interestingly, S3 transport worked fine with the avatar, thumb, and internal data config directives. Time to see if something changes with v8.2. No, and now my admin panel has thrown me out and I cannot get past the login window. Time to reload the backup and restart Apache.
I tried several iterations of this, including installing the upgrade from the
/src directory. That attempt said it was successful, then promptly put a light green "version mismatch" banner next to the addon. Same results... Reload.So, I am back to running the baseline profile and consulting with Dr. Kush for my hurting head.
The PHP directives are as follows:
PHP:
/* Flysystem AWS directive */
$config['fsAdapters']['data'] = function() use($s3)
{
return new \League\Flysystem\AwsS3v3\AwsS3Adapter($s3(), 'fotofora', 'data');};
/* Aws\S3\S3Client avatar & thumbnail url */
$config['externalDataUrl'] = function($externalPath, $canonical)
{
return 'https://fotofora.s3.us-east-1.amazonaws.com/data/' . $externalPath;};
/* AWS internal_data directory upload */
$config['fsAdapters']['internal-data'] = function() use($s3)
{
return new \League\Flysystem\AwsS3v3\AwsS3Adapter($s3(), 'fotofora', 'internal_data');
};
It says it's s3 compatible, so using the s3 info should work.
what is the actual errors you are seeing?
What version of XF are you on? this addon is no longer needed in the 2.3.x version. The plugin is 'native' included in the default xf codebase.
just need to add the config file directives to point your data and pdata folders to it.
what is the actual errors you are seeing?
What version of XF are you on? this addon is no longer needed in the 2.3.x version. The plugin is 'native' included in the default xf codebase.
just need to add the config file directives to point your data and pdata folders to it.
beerForo
Well-known member
I'm on 2.3 and I just noticed attachments are being stored locally. The s3 add-on is disabled. I didn't delete it yet, is that the step to get it to work? The reactions I uploaded did go to s3. Just not attachments.
Or maybe it's still working and the URL structure doesn't show amazon anymore? I do see some on amazon.
Confusing, but I guess it must be working as I don't even see an attachments folder on my server.
Or maybe it's still working and the URL structure doesn't show amazon anymore? I do see some on amazon.
Confusing, but I guess it must be working as I don't even see an attachments folder on my server.
Last edited:
Things from the
internal_data mount/bucket are streamed through your server, so the URLs will not have changed. This is no different with the add-on or in 2.3. Things from the data mount/bucket can be streamed directly, which is controlled via $config['externalDataUrl'] as detailed in the first post.