Chunked Uploads - XF2 [Paid] 1.0.2
- Thread starter JulianD
- Start date
JulianD
Well-known member
Can you post a screenshot of your attachment upload settings ?
The video / image buttons don't use
flow.js (they use upload functionality provided by Froala, see https://xenforo.com/community/threads/maximum-attachment-file-size-handling-is-inconsistent.208311/).So unless this Add-on has code to change this (this is possible but I don't know if it does that) chunked uploads will not work for them.
Last edited:
TCGMook
Active member
All good. It's all sorted out now. Everything is butter now.
JulianD
Well-known member
In that case to the normal uploading process is used and the file is not split in chunks.
That is correct
Writing chunks to a single temporary file as-is (instead of writing them as individual files one by one and reconstructing the original after upload has finished) could cut down rebuild time a bit.
But it would most likely not completely fix the issue as the temporary file would still have to be saved to backend storage as one chunk.
So if a file is 5 GB after rebuild, those entire 5 GB have to be synchronously uploaded to S3 after the upload has finished - and this can take some time depending on the connection between the server and the S3-compatible storage which in turn could lead to timeouts.
To fully solve this, it would probably be also necessary to upload to storage backend in chunks (instead of uploading the rebuild file as a whole).
And XenForo would have to fix a few inefficiencies as well to fully address this issue:
FFMPeg handling is very inefficient
\XFMG\Attachment\Media::onNewAttachment public function onNewAttachment(Attachment $attachment, \XF\FileWrapper $file) { /** @var \XFMG\Service\Media\TempCreator $tempCreator */ $tempCreator = \XF::app()->service('XFMG:Media\TempCreator')...
xenforo.com
Thumbnail / Poster handling seems inefficient
\XFMG\Attachment\Media::onAssociation if ($tempMedia->thumbnail_date) { $tempThumbPath = $tempMedia->getAbstractedTempThumbnailPath(); $mediaThumbPath = $container->getAbstractedThumbnailPath(); $tempFile = \XF\Util\File::copyAbstractedPathToTempFile($tempThumbPath)...
xenforo.com
Last edited:
Sirmadsen
Active member
That was over a year ago and highly doubt that it will come to the next major release. But i get it and understand what you are saying, why change something that already works...
benFF
Well-known member
Ah yes, that's a good point. I don't use off-site storage so I forgot about the re-uploading part.
aggrovated
New member
Gah! I paid for the Chunked Uploads add-on, but no download link was provided. Did you discontinue this add-on? I really wanted it, as this seems to be the only solution to uploading large files on Xenforo. really bummed out...and I lost $20 bucks also. 
JulianD
Well-known member
It is a manual process and it’s Sunday , I’m sorry for the inconvenience. The link should arrive in the next minutesGah! I paid for the Chunked Uploads add-on, but no download link was provided. Did you discontinue this add-on? I really wanted it, as this seems to be the only solution to uploading large files on Xenforo. really bummed out...and I lost $20 bucks also.
aggrovated
New member
Thanks Julian,
I have successfully implemented the addon, Thank you!
However, after testing various chunk size adjustments (131000kb, 13000kb and 1000kb) while using Cloudflare buckets. I am able to upload a 15mb image successfully, however a 39mb image fails altogether. I'm looking to upload more than 100mb images. I am aware of my servers config.php having a limitation also, and since have adjusted that to 1000000000kb. so I don't believe the bottleneck exist there. Is there a solution to my issue either you or someone can help me with please?
Edit: tried a 10000 chunk size. and boom! a 39mb image works.
I have successfully implemented the addon, Thank you!
However, after testing various chunk size adjustments (131000kb, 13000kb and 1000kb) while using Cloudflare buckets. I am able to upload a 15mb image successfully, however a 39mb image fails altogether. I'm looking to upload more than 100mb images. I am aware of my servers config.php having a limitation also, and since have adjusted that to 1000000000kb. so I don't believe the bottleneck exist there. Is there a solution to my issue either you or someone can help me with please?
Edit: tried a 10000 chunk size. and boom! a 39mb image works.
JulianD
Well-known member
I unfortunately do not have access to XFMG but it should work if there are no limitations with PHP (memory or timeout limits) or with the CDN/Proxy, just like you mentioned.
Do you plan to upload videos to XFMG? I might upload a 20GB video to an XenForo thread and report back my results to you if you'd like.
Yep, I do plan on uploading to XFMG; all are video files from a public event we hold. I'll pay the license fee and give it a whirl.
So I set the max upload size in PHP to be 25600M and see it reflected in phpinfo, but I'm still receiving a generic error from XF: "The uploaded file is too large for the server to process.". Is there another setting to set outside of upload_max_filesize?
Edit: Checked in the chunked upload options to eliminate the upload limits virtually and that enabled upload! Not sure if it's a security risk to leave this checked long-term though; we only want our admins to be able to upload crazy large files like this
Edit: Checked in the chunked upload options to eliminate the upload limits virtually and that enabled upload! Not sure if it's a security risk to leave this checked long-term though; we only want our admins to be able to upload crazy large files like this
Last edited:
