[bd] Attachment Store [Deleted]

FYI, storing attachments in s3 with cloudfront, works great with @Chris D 's XenMediaGallery.... Until I try to do any image manipulation (rotatate, crop, etc). Then I get this error, and the thumbnail is broken:

Code: 
Zend_Exception: Image /home/talkbass/html/internal_data/attachments/271/271901-d0d934c964b0308b74e4c9ac805186b6.data can not be found, try another image. -library/XenGallery/Helper/Image.php:43
Generated By: paul, 3 minutes ago
Stack Trace
#0 /home/talkbass/html/library/XenGallery/Model/Media.php(559): XenGallery_Helper_Image->__construct('/home/talkbass/...')
#1 /home/talkbass/html/library/XenGallery/ControllerPublic/Media.php(1728): XenGallery_Model_Media->rotateMedia(Array, -90)
#2 /home/talkbass/html/library/XenForo/FrontController.php(347): XenGallery_ControllerPublic_Media->actionRotate()
#3 /home/talkbass/html/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#4 /home/talkbass/html/index.php(13): XenForo_FrontController->run()
#5 {main}
Request State
array(3) {
["url"] => string(64) "http://www.talkbass.com/media/iphone4-003.236/rotate?clockwise=1"
["_GET"] => array(1) {
["clockwise"] => string(1) "1"
}
["_POST"] => array(0) {
}
}

Any media uploaded with "keep local copy" on, doesn't have this error, obviously. But I'd rather not keep local copies :)
 
kontrabass said:
FYI, storing attachments in s3 with cloudfront, works great with @Chris D 's XenMediaGallery.... Until I try to do any image manipulation (rotatate, crop, etc). Then I get this error, and the thumbnail is broken:

Code: 
Zend_Exception: Image /home/talkbass/html/internal_data/attachments/271/271901-d0d934c964b0308b74e4c9ac805186b6.data can not be found, try another image. -library/XenGallery/Helper/Image.php:43
Generated By: paul, 3 minutes ago
Stack Trace
#0 /home/talkbass/html/library/XenGallery/Model/Media.php(559): XenGallery_Helper_Image->__construct('/home/talkbass/...')
#1 /home/talkbass/html/library/XenGallery/ControllerPublic/Media.php(1728): XenGallery_Model_Media->rotateMedia(Array, -90)
#2 /home/talkbass/html/library/XenForo/FrontController.php(347): XenGallery_ControllerPublic_Media->actionRotate()
#3 /home/talkbass/html/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#4 /home/talkbass/html/index.php(13): XenForo_FrontController->run()
#5 {main}
Request State
array(3) {
["url"] => string(64) "http://www.talkbass.com/media/iphone4-003.236/rotate?clockwise=1"
["_GET"] => array(1) {
["clockwise"] => string(1) "1"
}
["_POST"] => array(0) {
}
}

Any media uploaded with "keep local copy" on, doesn't have this error, obviously. But I'd rather not keep local copies :)
Click to expand...
Are you running latest version of both add-on? This has been fixed a while iirc.
 
I made all the changes you suggested previously and they appear to work but apparently only if the files are stored locally.

I don't have a detailed understanding of how Attachment Store does its thing or any way of testing it. I don't think @MattW has had any issues, though, but I don't believe he uses S3 maybe the issue is exclusive to that.

Another customer did mention the same thing to me but they solved it by turning the local storage option on.

Either way, the issue was the same. Any image manipulation completes successfully but only the thumbnail breaks.

After the image is edited, the file hash changes, and the xf_attachment_data table is updated with the new hash. I save the new thumbnail to xengallery/data/ with the new file hash (different location but same naming convention as attachment thumbs).

So I'm guessing for some reason they don't get written to the cloud storage but I think we discussed that before and it didn't seem to be a problem.
 
Chris D said:
I made all the changes you suggested previously and they appear to work but apparently only if the files are stored locally.

I don't have a detailed understanding of how Attachment Store does its thing or any way of testing it. I don't think @MattW has had any issues, though, but I don't believe he uses S3 maybe the issue is exclusive to that.

Another customer did mention the same thing to me but they solved it by turning the local storage option on.

Either way, the issue was the same. Any image manipulation completes successfully but only the thumbnail breaks.

After the image is edited, the file hash changes, and the xf_attachment_data table is updated with the new hash. I save the new thumbnail to xengallery/data/ with the new file hash (different location but same naming convention as attachment thumbs).

So I'm guessing for some reason they don't get written to the cloud storage but I think we discussed that before and it didn't seem to be a problem.
Click to expand...
I will download the latest version and let you know.
 
Chris D said:
I made all the changes you suggested previously and they appear to work but apparently only if the files are stored locally.

I don't have a detailed understanding of how Attachment Store does its thing or any way of testing it. I don't think @MattW has had any issues, though, but I don't believe he uses S3 maybe the issue is exclusive to that.

Another customer did mention the same thing to me but they solved it by turning the local storage option on.

Either way, the issue was the same. Any image manipulation completes successfully but only the thumbnail breaks.

After the image is edited, the file hash changes, and the xf_attachment_data table is updated with the new hash. I save the new thumbnail to xengallery/data/ with the new file hash (different location but same naming convention as attachment thumbs).

So I'm guessing for some reason they don't get written to the cloud storage but I think we discussed that before and it didn't seem to be a problem.
Click to expand...
No, I use local storage, and don't have any issues
 
  • Like
Reactions: rdn
I have just installed this addon and cant upload files anymore:
The following error occurred
There was a problem uploading your file
Log:
XenForo_Exception: Failed to write the attachment file. - library/XenForo/DataWriter/AttachmentData.php:145

any idea?
 
Got this error on a clients server installing it for them:

Code: 
Error Info
Zend_Http_Client_Exception: Unable to read response, or response is empty - library/Zend/Http/Client.php:993
Generated By: fred13, 5 minutes ago
Stack Trace
#0 /home/nginx/domains/cointalk.com/public/library/Zend/Service/Amazon/S3.php(636): Zend_Http_Client->request('PUT')
#1 /home/nginx/domains/cointalk.com/public/library/Zend/Service/Amazon/S3.php(412): Zend_Service_Amazon_S3->_makeRequest('PUT', 'attachments.coi...', NULL, Array, '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...')
#2 /home/nginx/domains/cointalk.com/public/library/Zend/Service/Amazon/S3.php(454): Zend_Service_Amazon_S3->putObject('attachments.coi...', '\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x01\x00...', Array)
#3 /home/nginx/domains/cointalk.com/public/library/bdAttachmentStore/Model/File.php(164): Zend_Service_Amazon_S3->putFile('/tmp/phpnwpzxo', 'attachments.coi...', Array)
#4 /home/nginx/domains/cointalk.com/public/library/bdAttachmentStore/Model/File.php(28): bdAttachmentStore_Model_File->_s3_saveFile(Array, '/tmp/phpnwpzxo', '2014/06/293806_...', 'Italy 1916 2 Li...')
#5 /home/nginx/domains/cointalk.com/public/library/bdAttachmentStore/XenForo/DataWriter/AttachmentData.php(145): bdAttachmentStore_Model_File->saveFile('s3', Array, '/tmp/phpnwpzxo', '2014/06/293806_...', 'Italy 1916 2 Li...')
#6 /home/nginx/domains/cointalk.com/public/library/XenForo/DataWriter/AttachmentData.php(143): bdAttachmentStore_XenForo_DataWriter_AttachmentData->_writeAttachmentFile('/tmp/phpnwpzxo', Array)
#7 /home/nginx/domains/cointalk.com/public/library/bdAttachmentStore/XenForo/DataWriter/AttachmentData.php(63): XenForo_DataWriter_AttachmentData->_postSave()
#8 /home/nginx/domains/cointalk.com/public/library/XenForo/DataWriter.php(1409): bdAttachmentStore_XenForo_DataWriter_AttachmentData->_postSave()
#9 /home/nginx/domains/cointalk.com/public/library/XenForo/Model/Attachment.php(524): XenForo_DataWriter->save()
#10 /home/nginx/domains/cointalk.com/public/library/XenForo/ControllerPublic/Attachment.php(201): XenForo_Model_Attachment->insertUploadedAttachmentData(Object(XenForo_Upload), 20688)
#11 /home/nginx/domains/cointalk.com/public/library/XenForo/FrontController.php(347): XenForo_ControllerPublic_Attachment->actionDoUpload()
#12 /home/nginx/domains/cointalk.com/public/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#13 /home/nginx/domains/cointalk.com/public/index.php(13): XenForo_FrontController->run()
#14 {main}
Request State
array(3) {
  ["url"] => string(106) "http://www.cointalk.com/attachments/do-upload.json?hash=62ac977f92732a11a937bbb077c1c357&content_type=post"
  ["_GET"] => array(3) {
    ["/attachments/do-upload_json"] => string(0) ""
    ["hash"] => string(32) "62ac977f92732a11a937bbb077c1c357"
    ["content_type"] => string(4) "post"
  }
  ["_POST"] => array(7) {
    ["Filename"] => string(32) "Italy 1916 2 Lire AU version.jpg"
    ["_xfNoRedirect"] => string(1) "1"
    ["content_data"] => array(1) {
      ["post_id"] => string(7) "1935298"
    }
    ["_xfToken"] => string(8) "********"
    ["_xfSessionId"] => string(32) "08973ab39c5a9a651189e5a8f20e6f26"
    ["_xfResponseType"] => string(4) "json"
    ["Upload"] => string(12) "Submit Query"
  }
}
 
Also can you explain in more detail the following:

We have 62GB of attachments we would like to move to S3. Right now we are uploading attachments directly to S3 and would like it to stay that way.

I notice at the bottom of the cache rebuild page there are the following:

Move Attachment Data

and

Update Attachment Data Storage Options

I am not sure which one I need or do I need both?
 
So, I changed my attachment store options from using s3 + cloudfront to just s3 (no cloudfront). All of my attachments look ok in the forums, but in @Chris D Xen Media Gallery, thumbnails for all NEW media uploads are broken... until I rebuild them, then they're fine again. Old media (pre-change) are all ok. Is there something I can do?
 
"in External Data directory"

So, I just want to store my attachment files on another drive on my dedicated server (I'm moving all of my data to one drive for centralization and easier backup).

I've found instructions for migrating to S3 and incorporating FTP for upload to another server, but how do I direct the addon to another drive/directory on my existing server?

And then, how do I migrate the files?
 
Main Company said:
"in External Data directory"

So, I just want to store my attachment files on another drive on my dedicated server (I'm moving all of my data to one drive for centralization and easier backup).

I've found instructions for migrating to S3 and incorporating FTP for upload to another server, but how do I direct the addon to another drive/directory on my existing server?

And then, how do I migrate the files?
Click to expand...
Anyone?
 
If it's on the same server, I'd just create a symlink from the 2nd drive to the location of the forum data directory. That's what I'm doing when serving them via a different domain
Code: 
[root@host public]# ls -hl
total 36K
-rw-r--r-- 1 nginx nginx 1.6K Jul 23 12:48 403.html
-rw-r--r-- 1 nginx nginx 1.6K Jul 23 12:48 404.html
-rw-r--r-- 1 nginx nginx 2.1K Jul 23 12:48 500.html
-rw-r--r-- 1 nginx nginx 2.1K Jul 23 12:48 502.html
-rw-r--r-- 1 nginx nginx 2.2K Jul 23 12:48 503.html
-rw-r--r-- 1 nginx nginx 2.1K Jul 23 12:48 504.html
-rw-r--r-- 1 nginx nginx 2.2K Jul 23 12:48 50x.html
lrwxrwxrwx 1 root  nginx   44 Jul 23 12:59 data -> /home/nginx/domains/z22se.co.uk/public/data/
-rw-r--r-- 1 nginx nginx 1.2K Jul 23 12:48 index.html
lrwxrwxrwx 1 root  nginx   42 Jul 23 13:00 js -> /home/nginx/domains/z22se.co.uk/public/js/
-rw-r--r-- 1 nginx nginx 2.2K Jul 23 12:48 maintenance.html
lrwxrwxrwx 1 root  nginx   46 Jul 23 12:59 styles -> /home/nginx/domains/z22se.co.uk/public/styles/
[root@host public]#
 
Okay, so I asked our technical person to step-in to help implement this and he set it up to store the files on another drive, within the same server, using the FTP functionality (as shown in the image below).

My understanding is that before we enabled BD Attachment addon, the way XF handled an attachment was to:
  1. Store the attachment in xenforo/data/accachments directory in xxxx.data format.
  2. Add a hash record in XF DB, where this hash record keeps the mapping between the real file name and the xxxx.data file.
Now, when we add a new attachment, it will be directly put into folder: /data2/xenforo/data, with the filename: {id}_{filename}.{extension}, and a hash record in the XF DB will still be created.

My question is: how do we migrate the attachments that we uploaded to XF, before we enabled BD Attachments Addon, to the new location?

xf_attachment_config_blurred.webp
 
xfrocks said:
It is reversible. There is a tool included to migrate attachments across different storage options. You can go from default to S3 and back as many times as you want.
Click to expand...
Okay, I just found the above post. But where is the tool? What did I miss? Thanks.
 
You must log in or register to reply here.

Similar threads

Coop1979
  • Question Question
XF 2.2 Help creating Nginx url redirects from BD [Attachment Store] to default url schema
Replies
0
Views
256
Coop1979
Coop1979
Zig
XF 2.1 XG2XFMG importer with [bd]AttachmentStore
Replies
0
Views
367
Zig
Zig
ratrodbikes
  • Question Question
XF 2.1 ATTACHMENT STORE KILLING ME!
Replies
2
Views
633
Ozzy47
Ozzy47
ratrodbikes
  • Question Question
XF 2.1 ABOUT TO UPGRADE XF1 to XF2 - QUESTION ABOUT ATTACHMENT STORE!
Replies
3
Views
679
ratrodbikes
ratrodbikes
xfrocks
[bd] Medal [Deleted]
6 7 8
Replies
151
Views
16K
Ozzy47
Ozzy47
Top Bottom