Using DigitalOcean Spaces or Amazon S3 for file storage

Using DigitalOcean Spaces or Amazon S3 for file storage

No permission to download
Overview Updates (6) Reviews (7) Discussion
Sim said:
What version of XF are you currently running?
Click to expand...

Not my forum, but it is one I'm setting this up for:
MySiteGuy said:
It's XF 2.2.17, using PHP 8.1.31. I'm getting numerous Guzzle related class not found errors trying to load every page. I've disabled the add-on and commented out the code in config.php until I can resolve it. Was there something changed in 2.2.17?
Click to expand...
 
I literally used the add-on on 2.2.17 last week with no issues. I’m pretty sure those Guzzle classes shouldn’t be there.

I recommend:

Commenting out the config, uninstalling the add-on, deleting all of its files (preferably the XFAws folder directly from the server), reinstalling version 2.4.0 of the add-on and re-enabling the config and trying again.
 
Chris D said:
Just checked this class and function and all is present and correct and as it should be. This leads me to believe you may have an add-on which itself is including a different version of Guzzle to the one we ship.

So first step is to comment out the config.php stuff, disable all add-ons, re-enable the add-on, uncomment the config, and test again. I’m guessing it will work. Then re-enable each add-on to figure out which one causes the issue.
Click to expand...

@Chris D the 2.4.0 version of this addon ships with psr/http-message v2.0, while XenForo 2.2.x ships with psr/http-message v1.0.1

Addon v2.4.0:

JSON: 
            "name": "psr/http-message",
            "version": "2.0",

XenForo v2.2.x:

JSON: 
             "name": "psr/http-message",
            "version": "1.0.1",

The difference between the two being the function definition in the interface:

PHP: 
@@ -37,7 +38,7 @@ interface UriInterface
     * @see https://tools.ietf.org/html/rfc3986#section-3.1
     * @return string The URI scheme.
     */
    public function getScheme();
    public function getScheme(): string;

I believe that this is what is causing the conflict.

I note that v2.3.0 of this addon did not ship with psr/http-message at all, so would not have caused any issues.

Can I suggest that you re-release this addon with psr/http-message locked to v1.0.1 for compatibility with XF 2.2.x ?
 
Hi, short question.
The ressource mentioned that S3 is supported in XF 2.3 without plugin?

xenforo.com

Using DigitalOcean Spaces or Amazon S3 for file storage in XF 2.1 & XF 2.2 - Updates

xenforo.com xenforo.com

Is there an official documentation on how to set it up? Or is the setup the same as with this plugin? I didn't find anything in the Admin Panel nor in the xenforo documentation.

Is there some guide how I would do this on an existing forum with xf 2.3? I would appreciate your help.

Thanks.
 
The steps in the resource are the same for both 2.2 and 2.3. It’s just for 2.3 you don’t have to install the add-on.
 
Chris D said:
The steps in the resource are the same for both 2.2 and 2.3. It’s just for 2.3 you don’t have to install the add-on.
Click to expand...
Is there a guide or tutorial on how to build or use Amazon S3/Minio on XenForo Resource Manager without a plugin?

Can Minio work with this?

I'm sorry for asking again
 
Not sure what Minio is but if it has an S3 compatible API then yes, that will work.

To clarify, this isn't a "plugin" exactly - for 2.2 it just makes the Amazon S3 SDK available which is the necessary code for communicating with S3 compatible APIs.

The only difference between 2.2 and 2.3 is that the SDK (a much smaller version) is included by default in XF 2.3.
 
Chris D said:
Not sure what Minio is but if it has an S3 compatible API then yes, that will work.

To clarify, this isn't a "plugin" exactly - for 2.2 it just makes the Amazon S3 SDK available which is the necessary code for communicating with S3 compatible APIs.

The only difference between 2.2 and 2.3 is that the SDK (a much smaller version) is included by default in XF 2.3.
Click to expand...
Thank you for the explanation!

Could you take a look at this?
github.com

GitHub - minio/minio: MinIO is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license.

MinIO is a high-performance, S3 compatible object store, open sourced under GNU AGPLv3 license. - minio/minio
github.com github.com

Minio (README):
MinIO is a High Performance Object Storage released under GNU Affero General Public License v3.0. It is API compatible with Amazon S3 cloud storage service. Use MinIO to build high performance infrastructure for machine learning, analytics and application data workloads. To learn more about what MinIO is doing for AI storage, go to AI storage documentation.

Because I would like to build a storage host server to use Minio without Amazon S3. Not sure if that will work with XenForo 2.3.0 and to up 2.3.5
 
Sim said:
@Chris D the 2.4.0 version of this addon ships with psr/http-message v2.0, while XenForo 2.2.x ships with psr/http-message v1.0.1

Addon v2.4.0:

JSON: 
            "name": "psr/http-message",
            "version": "2.0",

XenForo v2.2.x:

JSON: 
             "name": "psr/http-message",
            "version": "1.0.1",

The difference between the two being the function definition in the interface:

PHP: 
@@ -37,7 +38,7 @@ interface UriInterface
     * @see https://tools.ietf.org/html/rfc3986#section-3.1
     * @return string The URI scheme.
     */
    public function getScheme();
    public function getScheme(): string;

I believe that this is what is causing the conflict.

I note that v2.3.0 of this addon did not ship with psr/http-message at all, so would not have caused any issues.

Can I suggest that you re-release this addon with psr/http-message locked to v1.0.1 for compatibility with XF 2.2.x ?
Click to expand...

@Chris D we're running into this same issue as well, is this something that is going to be addressed in a release?

I've updated it on a version of the build we have, but that isn't going to help other people, I'd be happy to upload that if needed so you can update the release here. I'd imagine this add-on should be marked as unmaintained though if issues aren't being addressed
 
Last edited:
The resource is maintained, but the add-on is not. It's no longer needed for XenForo 2.3, but the guide is relevant.

The AWS SDK that was shipped in the "add-on" no longer supports below PHP 8.1 so I'm re-releasing today pinned to the last versions supported that will be usable by XenForo 2.2.

There will be no further releases after this and the recommendation is to upgrade to XenForo 2.3.

Thank you @Sim for discovering the issue.
 
Last edited:
XenForo updated Using DigitalOcean Spaces or Amazon S3 for file storage in XF 2.1 & XF 2.2 with a new update entry:

Final release for XenForo 2.2

Due to an issue, we are releasing a new version of the add-on to support customers running XenForo 2.2 and older versions of PHP.

This is the last supported release due to XenForo 2.2 no longer being supported and this being the last version of the AWS SDK that can be supported by XenForo 2.2.

We continue to recommend the following:
  • We recommend upgrading to XenForo 2.3 at the earliest opportunity
  • This add-on can be safely removed if you are running XenForo 2.3 as this ships...
Click to expand...

Read the rest of this update entry...
 
I'm getting some exceptions randomly since setting this up.

Code: 
InvalidArgumentException: Must pass valid resource in src/XF/Http/ResponseStream.php:18
Generated by: Unknown account Apr 11, 2025 at 11:46 AM
Stack trace
#0 src/XF/Http/Response.php(262): XF\Http\ResponseStream->__construct(false, 107238)
#1 src/XF/Pub/View/Attachment/View.php(28): XF\Http\Response->responseStream(false, 107238)
#2 src/XF/Mvc/Renderer/AbstractRenderer.php(92): XF\Pub\View\Attachment\View->renderRaw()
#3 src/XF/Mvc/Renderer/Raw.php(39): XF\Mvc\Renderer\AbstractRenderer->renderViewObject('XF:Attachment\\V...', '', Array)
#4 src/XF/Mvc/Dispatcher.php(471): XF\Mvc\Renderer\Raw->renderView('XF:Attachment\\V...', '', Array)
#5 src/XF/Mvc/Dispatcher.php(453): XF\Mvc\Dispatcher->renderView(Object(XF\Mvc\Renderer\Raw), Object(XF\Mvc\Reply\View))
#6 src/XF/Mvc/Dispatcher.php(412): XF\Mvc\Dispatcher->renderReply(Object(XF\Mvc\Renderer\Raw), Object(XF\Mvc\Reply\View))
#7 src/XF/Mvc/Dispatcher.php(66): XF\Mvc\Dispatcher->render(Object(XF\Mvc\Reply\View), 'raw')
#8 src/XF/App.php(2826): XF\Mvc\Dispatcher->run()
#9 src/XF.php(806): XF\App->run()
#10 index.php(23): XF::runApp('XF\\Pub\\App')
#11 {main}

I saw this reported earlier in thread, but there doesn't seem to be any consensuses about what todo about it.
 
mastahg said:
I'm getting some exceptions randomly since setting this up.

Code: 
InvalidArgumentException: Must pass valid resource in src/XF/Http/ResponseStream.php:18
Generated by: Unknown account Apr 11, 2025 at 11:46 AM
Stack trace
#0 src/XF/Http/Response.php(262): XF\Http\ResponseStream->__construct(false, 107238)
#1 src/XF/Pub/View/Attachment/View.php(28): XF\Http\Response->responseStream(false, 107238)
#2 src/XF/Mvc/Renderer/AbstractRenderer.php(92): XF\Pub\View\Attachment\View->renderRaw()
#3 src/XF/Mvc/Renderer/Raw.php(39): XF\Mvc\Renderer\AbstractRenderer->renderViewObject('XF:Attachment\\V...', '', Array)
#4 src/XF/Mvc/Dispatcher.php(471): XF\Mvc\Renderer\Raw->renderView('XF:Attachment\\V...', '', Array)
#5 src/XF/Mvc/Dispatcher.php(453): XF\Mvc\Dispatcher->renderView(Object(XF\Mvc\Renderer\Raw), Object(XF\Mvc\Reply\View))
#6 src/XF/Mvc/Dispatcher.php(412): XF\Mvc\Dispatcher->renderReply(Object(XF\Mvc\Renderer\Raw), Object(XF\Mvc\Reply\View))
#7 src/XF/Mvc/Dispatcher.php(66): XF\Mvc\Dispatcher->render(Object(XF\Mvc\Reply\View), 'raw')
#8 src/XF/App.php(2826): XF\Mvc\Dispatcher->run()
#9 src/XF.php(806): XF\App->run()
#10 index.php(23): XF::runApp('XF\\Pub\\App')
#11 {main}

I saw this reported earlier in thread, but there doesn't seem to be any consensuses about what todo about it.
Click to expand...
For me Snuffulupagus caused it. I had to add
sp.wrappers_whitelist.list("file,php,phar,http,https");

to my config
 
Hello. I just purchased your plugin and found out that the setup isn't so simple. Please tell me, if I connect S3 storage according to the instructions for XenForo 2.3 (Resource Manager), will only the site attachments (including resources) be uploaded to S3, or will everything be uploaded (including avatars and so on)? I would like to configure it so that only resources are stored on S3, or at least resources and post attachments, if that's easier to do.
 
AndryOsminin said:
Hello. I just purchased your plugin and found out that the setup isn't so simple. Please tell me, if I connect S3 storage according to the instructions for XenForo 2.3 (Resource Manager), will only the site attachments (including resources) be uploaded to S3, or will everything be uploaded (including avatars and so on)? I would like to configure it so that only resources are stored on S3, or at least resources and post attachments, if that's easier to do.
Click to expand...
I figured it out, connected only the internal_data switch. As I understand, it is a bit difficult to make it so that in S3 there is only the internal_data/attachments folder. I wonder if it is possible to store cache folders in S3 storage for fast forum action.

And another question, does S3 object storage need to be made private? It's just that my S3 storage has a CDN option for public object storage.
 
You must log in or register to reply here.

Similar threads

B
Using Digital Ocean Spaces or Amazon S3 for internal_data / attachments
Replies
3
Views
1K
Vlady
V
Back
Top Bottom