MG 1.1 Members and guests can view deleted media using full URL

[Sim]

We have an issue on ZooChat where members and guests can view deleted media using the /full url

I checked on my other sites running XF1.5/MG1.1 and my dev server and also on my XF2 dev server and they all work as expected (soft deleted photos are not visible to members/guests) - so there's something about how ZooChat is set up which is different.

The main difference between all these servers is that ZooChat imported its media content from PhotoPost, while none of the other sites I've tested did.

EDIT: I just tested another site which had media imported from PhotoPost and this one works as expected - so I'm not sure its directly related to the import? Unless there were some permissions imported as well during the import process which were unique to ZooChat?

So otherwise, there is something specific about the configuration on ZooChat which causes this issue, but I can't work out which setting is causing this behaviour.

Example: this following image has been (soft) deleted - media link: https://www.zoochat.com/community/media/test.401362/), and yet the image itself is still visible to guests (and members) using the direct URL - "full" image url: https://www.zoochat.com/community/media/test.401362/full

 

[Chris D]

Example: this following image has been (soft) deleted - media link: https://www.zoochat.com/community/media/test.401362/), and yet the image itself is still visible to guests (and members) using the direct URL - "full" image url: https://www.zoochat.com/community/media/test.401362/full

Neither of those links are visible to me, both giving the expected no permission error. Are you sure the permissions are set up correctly here, or is it intermittent, somehow?

 

[Sim]

Hmm ... I tried it on my phone in incognito mode using 4G I get the "You do not have permission to view deleted media" error.

However, when swapping back to WiFi, I can view it the image.

Very strange - will investigate further and report back.

 

[Sim]

Hmm - even more strange.

I've narrowed the issue down to my office network. When my other machine (a laptop) is online via office network WiFi, it can see the deleted image. When it's tethered to my phone and thus using 4G, it gets the error message.

I thought it may have been an IPv6 related issue, but turning off IPv6 in my router didn't change anything - still happened on IPv4.

So I'm investigating my settings now to see if there's something I've got tied to my local network which may be overriding something. It's not related to my machine - happens on other machines on my network too.

From what I can tell, my nginx config and my Cloudflare config is identical between ZooChat and PropertyChat, but it happens on one site and not the other.

 

[Sim]

I've had confirmation that my moderators can also see the image that they should not be able to see - even when logged out of the forums and viewing as guests.

It could be that there is some kind of caching issue happening here.

However, I tried turning off Cloudflare and forcing a page refresh, but still got the same result with a 200 status response.

@Chris D ... I was trying to debug on my live site, but did not seem to be able to find the correct function to insert some debugging code into.

Am I correct in that the entry point for "full" images is XenGallery_ControllerPublic_Media::actionFull ?? Or have I missed something?

 

[Sim]

I just tried using wget on one of my development VPS servers to access that URL and it successfully downloaded the png file where it should have received an error.

I'm thinking it's either a Cloudflare cache issue, or a problem with the XFMG permissions.

 

[Chris D]

It's almost certainly a Cloudflare cache issue. There are headers which are suggesting that the cache is being hit:

cf-cache-status: HIT

I'm now seeing the image, when I shouldn't be, even though when I checked this morning, I still wasn't able to. That's not something that can be caused by permissions, unfortunately and with the positive confirmation of CF caching involved then that would seem to be the target.

 

[Sim]

Yup - I've confirmed it's Cloudflare.

I paused Cloudflare on my site, uploaded a new image, checked I could access the full image, deleted the image, tried to access the full image again and got the expected error.

Once I turned Cloudflare back on again, I continued getting the error. Interestingly I also got the error on my phone whereas previously I was able to view it - although my VPS dev server is still getting the image successfully using wget.

I tested the same steps on xenforo.com and got the expected error - but also noticed the cache headers were set to Cache-Control: private, no-cache, max-age=0 - so it's clear that you don't explicitly cache media gallery images in Cloudflare from xenforo.com, whereas I do have a cache directive.

I'm going to investigate using the Cloudflare API to invalidate the cache when an image is deleted (reference: https://api.cloudflare.com/#zone-purge-files-by-url).

 

[Sim]

@Chris D just out of curiosity - what is the purpose of the ?d={$media.last_edit_date} query string added to some links to "full" media files in the gallery?

Is that just a general cache-busting mechanism?

 

[Chris D]

Yeah, it's a cache buster.

 

[Sim]

Just to confirm my findings - I found an image on ZooChat that had been widely viewed but recently deleted and confirmed that it was still serving the image from the Cloudflare cache to guest users.

I then used the Cloudflare control panel to invalidate the full url of the image and waited a while (they suggested a minimum of 30 seconds), before reloading the page. It gave me the error code I was hoping for - manual cache invalidation worked.

Next step - implement automatic cache invalidation.

Thanks for your help @Chris D

 

[Sim]

Actually that's not quite going to work.

Any time a moderator views a deleted image, it's back in the cache.

Also, turns out that Cloudflare caches 403 & 404 responses for 5 minutes, so if a guest views a deleted image full URL and gets a 403 or 404, then the moderators will also get the same error viewing the file.

I wonder if showing a unique URL to moderators would help here? Probably better to set a custom cache header (no cache for moderators, full cache for guests with auto cache invalidation upon deletion)