Add-on Amazon S3 file uploads

[Bill.D]

I don't suppose you could post new instructions then?

 

[Bill.D]

Hey Mike,

When you say to Add these settings to the config, Where are you referring? Its not config.php because I get an error when I do that.

    'internalDataPath' => 's3://your-bucket-name/internal_data',
    'externalDataPath' => 's3://your-bucket-name/data',
    'externalDataUrl' => 'https://s3.amazonaws.com/your-bucket-name/data',

Thanks,
-Bill

 

[Jeremy]

How did you add them to config? Just like that? Or did you use the proper $config['internalDataPath'] syntax?

 

[Bill.D]

lol.. No.. Just like that.. I am a n00b.. Sorry.. Thanks for the hint!!

 

[Bill.D]

Ok, So another n00b question. Does the bucket need to be set to public Read, or does it use the Keys I provided? I was getting access denied so I set it to public and that worked. But I am wondering if it shouls be set to use the key?

Thank!
-Bill

 

[Bill.D]

Is there a way to provided an external like for the full sized attachments? I want to use a CDN where I can and when I entered the CDN address in the "externalDataUrl" I found that when I changed my Avatar that while the image did upload; it was named the same and what showed was the old cached image.

 

[Bill.D]

Hey All,

I am getting a weird error. I am currently using amazon s3 for my DATA & INTERNAL_DATA folders. So far everything has been prefect. But now, when we upload a moderately sized photo; it shows up distorted.

Here is an example:

Here is the same .data file that I changed to .jpg that was downloaded from the S3 bucket:


Why is the file from S3 getting distorted by Xenforo?

Thanks,
-Bill

 

[Bill.D]

Hey Mike,

I've got a lot of this working but I am having an issue with Images. Can you take a look at my thread and see if you would know whats causing that?
http://xenforo.com/community/threads/images-amazon-s3-streamwrapper-corrupt-distorted-images.59610/
Note: Its only distorted after it gets process by Xenforo and shown, you can see from the good image that I Downloaded from S3 directly that is is fine.

 

[Jeremy]

How are you uploading them to Amazon S3?

 

[Bill.D]

They get upload by Xenforo via the instructions on this thread: http://xenforo.com/community/threads/amazon-s3-file-uploads.47008/#post-572526
The upload seems perfect. You can see the above good image.. It was the .data file from the s3 bucket.

 

[Jeremy]

You should either ask the author for help (as he's written the upload to S3 code) or contact Amazon to see if they know why it is corrupting (if its on their end).

 

[Bill.D]

Yes, I did post in there as well.

 

[Marcus]

"Zend_Service_Amazon was removed in ZF2. you would need to use ZF1 in order to use this class."
http://framework.zend.com/manual/1.10/en/zend.service.amazon.s3.html

There is this in Zend 2.1 but it looks like it only fetches items from amazons store:
http://framework.zend.com/manual/2.1/en/modules/zendservice.amazon.html


This is the only error I encountered with ZF1: Fetch public templates as files returns Parse Error for Amazon S3 Location

 

[Sim]

Just a comment on using S3 to serve files for your websites - while it is quite possible to serve files directly from S3, you might find that performance is less than stellar.

S3 is intended for bulk data storage and occasional file access, not for serving website data from directly - it's not optimised to do so.

You really want to put a CDN in front of S3 to make file serving perform well - this can be pretty much seamless with CloudFront, but there are possibly other solutions as well.

You would then just change the externalDataUrl from Mike's config to use your CDN URL to serve external data from - for example:

    'internalDataPath' => 's3://your-bucket-name/internal_data',
    'externalDataPath' => 's3://your-bucket-name/data',
    'externalDataUrl' => 'https://media.example.com/',

... where media.example.com is the CNAME you have set up for your website to serve data from CloudFront (alternatively, rather than use a CNAME, use CloudFront URLs directly).

 

[Marcus]

Amazon S3 can also receive uploads with PUT and POST by http. An s3 wrapper wouldnt be needed then. Thats kind of new and you would need to tell amazon some additional data in addition to these POST or PUT requests. Does someone figure it out?

http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingHTTPPOST.html

__________
Browser-Based Uploads Using POST (AWS Signature Version 2)
Authenticating Requests in Browser-Based Uploads Using POST (AWS Signature Version 4) in the Amazon Simple Storage Service API Reference.



The following figure shows an upload using Amazon S3 POST.

Uploading Using POST

1 The user opens a web browser and accesses your web page.
2 Your web page contains an HTTP form that contains all the information necessary for the user to upload content to Amazon S3.
3 The user uploads content directly to Amazon S3.

Note

Query string authentication is not supported for POST.

________________


Either its possible without these additional data configured in php, but just use nginx to add this data to the PUT requests. Or it might be possible to extend the existing http stream wrapper. Or it might be possible without doing anything like that.

That now supported http method should be faster than using a stream wrapper.