Not planned Big problem: XenForo duplication of thumbnails

[imthebest]

Hi,

When Xen Media Gallery wasn't part of XenForo this was somehow understandable but now that XFMG is part of XenForo I think this is totally unacceptable.

XFMG stores all the thumbnails that display inside the gallery within data/xengallery. The problem is that XenForo *stores another exact copy of every thumbnail* at data/attachments. This is a big problem for large galleries:

#du -h data/xengallery
2.6GB    xengallery
#find data/xengallery -type f | wc -l
41,052 files

#du -h data/attachments
2.6GB    attachments
#find data/attachments -type f | wc -l
40,219 files

So I'm storing 40,219 files which makes a total of 2.6GB for nothing. This gets even worst when it comes to backups because you're complicating the process and making it more resource intensive.

Again, this is unacceptable and should be addressed in the near future.

Thanks,
Super120

 

[Martok]

There is a good reason why the thumbnails are stored in two locations and @Chris D explained it here:

https://xenresources.com/threads/media-gallery-not-honoring-xfs-thumbs-size-settings.1809/#post-9423

As for "it's not acceptable..." stuff and your current bee in your bonnet with picking holes in XFMG, you do need to bear in mind that Chris has only been an XenForo developer since November. Also, the primary focus I would guess for the XenForo team is developing XenForo 2.0.

So by all means point out bugs that need immediate attention but do calm down as I, and many others, would rather have the team focused on working ot get out XF 2.0 as soon as they can.

 

[imthebest]

(...)and your current bee in your bonnet with picking holes in XFMG(...)

I'm not here all the time, I like anyone here have a real life. The last few months my participation here was low. I only have time to test the software and report bugs from time to time. Right now I had a lot of work pending on my forums and I'm busy working on it, and as a result of this work I'm finding bugs.

I'm not trying to be rude nor blame Chris, I'm just finding bugs and reporting them. I'm also finding things that I consider should (or must) improve and creating the thread here. It's up to Chris to take my suggestions or not.

I have a lot of respect for Chris and I consider XFMG to be an awesome add-on. I also respect you Martok because you're a well known member here at XenForo.com

So it's all fine, take it easy and let's work together to help the devs improve this wonderful software!

 

[imthebest]

I see that my suggestion has been tagged as "No Thanks". At least I'd like to know why the devs consider that this isn't a problem. I mean... we are talking about content duplication with no use and we are talking about a lot of files! Honestly and with all due respect, I don't understand.

Unless someone explains me why this happens, to me it's a design issue.

 

[Martok]

we are talking about content duplication with no use

Read the post I linked to, it explains it in there.

 

[Mike]

You were linked to an explanation. They are different files used in different contexts and there certainly is a use with how the system is designed. Your belief that they should not be separated does not account for all the scenarios and the underlying architecture of the system. If you wish to violate these constraints for your use to save some space, you may wish to post a custom development request to have someone develop something designed specifically for your needs.

 

[imthebest]

I have read these 2 sources from Chris:

The reason there are two thumbnails is for an important reason.

The fact that they are created equal doesn't help what might happen in the future.

What I am referring to is the ability in the Admin CP to Rebuild Attachment Thumbnails. Xen Media Gallery uses the normal XF attachment system and XMG creates a new content type called "xengallery_media". The aforementioned rebuilder in the Admin CP doesn't care about the content type. So when it runs, it rebuilds all thumbnails in data/attachments to the same setting as the Attachment Options in the Admin CP. So, if that was the only place XMG thumbnails were stored and you ran the rebuild it would resize all the gallery thumbs too.

So, it just doesn't work.

And because XMG uses its own system to create thumbnails, and enables you to set a width and a height instead of just a max width, it's important that it creates its own thumbnails, and it also enables the Gallery settings to not have an impact on thumbnails shown for attachments in posts.

https://xenresources.com/threads/media-gallery-not-honoring-xfs-thumbs-size-settings.1809/#post-9423

data/xengallery is the directory which will house the actual thumbnails that display inside the gallery.

data/attachments is the directory where all XenForo attachment thumbnails are stored. XenForo does store another copy of the XMG thumbnails here. This is a XF behaviour that is mostly unavoidable.

https://xenforo.com/community/threa...enforo-paid-deleted.49067/page-97#post-843561

So the one million dollar question to me is: if XF stores another copy at data/attachments of the XFMG thumbnails originally stored on data/gallery, why XF does this? What is the use for these files? Chris said this:

The one in the attachments directory is never used so it is not necessary to keep it the same size.

https://xenresources.com/threads/media-gallery-not-honoring-xfs-thumbs-size-settings.1809/#post-9451

But now Mike said:

They are different files used in different contexts and there certainly is a use with how the system is designed.

So I'm confused. What is the use for these copied thumbnails located at data/attachments?

 

[Martok]

It's not hard really, especially as Chris explained it in his post.

Thumbnails in data/attachments are used for thumbnails in posts.
Thumbnails in data/gallery are the thumbnails used in XFMG.

These thumbnails can and usually are different sizes, with the ones in posts typically being much smaller than the ones in XFMG.

 

[Mike]

Thumbnails in data/attachments are used for thumbnails in posts.

While correct, it's more correct to say "thumbnails in data/attachments should be used as thumbnails for any attachment regardless of content type". If you want custom thumbnails for your attachment for a specific content type, you shouldn't touch these; you store your own. (There is actually an optimization in XFMG that breaks this, which will change depending on performance; rebuilding XF's attachments thumbs would undo that anyway.)

For anyone that has delved into the structure of the attachment system, it should be pretty clear what the intention of the design is (reuse), even if it isn't exposed in the UI at this time (though there are cases that do this internally). "Never used" may be the case now, though I'm rather surprised an add-on hasn't popped up that provided an attachment browser.