XF 1.5 Image proxy causes problems with Flickr images

[Stuart Wright]

With the image proxy turned on, several of the Flickr hosted images in posts don't show. Little broken X image and link.
E.g. https://www.avforums.com/threads/nikon-photo-sharing.1804393/page-55



With the image proxy turned off, they are all fine.


We recently started using Cloudflare. This problem has started since then. Don't know if that's relevant.

Please help.

 

[Martok]

It's likely an issue with Cloudflare. I've just visited your site and that image is showing for me:



I did notice some other Flickr images weren't showing but once I refreshed the page they loaded correctly.

I believe you are still on XenForo 1.5.x so any Flickr media sites would probably have come from an add-on (s9e Media Sites?) so any other issues would likely relate to that (though I still think it's something related to Cloudflare).

 

[Stuart Wright]

Thank you

I did notice some other Flickr images weren't showing but once I refreshed the page they loaded correctly.

Yes it seems to be random which load and which fail and different with each page load.
I don't have a Flickr media site addon.
I'll look into the Cloudflare possibility.

 

[DeltaHF]

I'm glad I found your thread, @Stuart Wright. I have experienced the same thing. I noticed it can affect images with Imgur as well, but it may be even more widespread — we just don't notice it from other origins because people don't use them as much.

Are you using Cloudflare Page Rules to cache your Image Proxy images? Are you using a second proxy server to fetch your XF image proxy images, so your "real" server's IP address doesn't leak?

I have been doing a lot of research into the problem, and here's what seems to be happening:

1. For whatever reason, the request to the origin server fails, XenForo returns the broken image placeholder, and your local Cloudflare cache saves that as the image (if you have a Page Rule configured to cache XenForo's proxy.php images).
2. If another visitor, who's accessing your site through a different Cloudflare edge server, requests that same image, it won't be in their local Cloudflare cache (unless you're using Argo), so Cloudflare makes another request to your server for the image.
3. If the proxied image is old enough that XenForo decides it will try to re-fetch it, it will do that, and the request will usually succeed. XenForo then serves the full image back to this latest request from Cloudflare and that edge caches it.
4. Now one Cloudflare edge cache location has the image placeholder, and the other cache location has the real image. This is why you and some of your visitors might see the broken image, but @Martok sees it correctly (you can see which Cloudflare edge cache you're accessing by adding "/cdn-cgi/trace" to the end of your Cloudflare-hosted domain, e.g. https://xenforo.com/cdn-cgi/trace).
5. Flushing your Cloudflare cache will temporarily force every edge location to re-fetch the image, and it will fix the problem... but only temporarily. Keep browsing your image-heavy threads and it will happen again.

I haven't yet figured out a way around this problem. The key seems to be figuring out how to get Cloudflare to invalidate or not cache the XenForo broken image placeholder...

 

[DeltaHF]

Unfortunately, Flickr (and ImageShack, to an extent) has continued to cause problems for my community, which regularly shares large batches of photos from the platform. These large "photo dumps" appear to be the most likely instance that triggers the issue, and I was even able to replicate it here in the test forums (you won't see broken images now, until the proxy is cleared and XF.com has to refetch them, but some were broken when I initially posted them).

Cloudflare is not the cause of the issue, but it can exaggerate the problem. If you have a Page Rule configured to "cache everything" from proxy.php, it will cache XenForo's broken image placeholder image. The only way I have found around that issue is by using Cloudflare Workers, which I plan on posting as a resource for future reference, but it does not solve the root of the problem, which is the fact that Flickr is behaving very badly with the Image Proxy these days and there doesn't appear to be anything we can do about it short of disabling the Image Proxy completely.

 

[empire]

Was there any solution to this issue that doesn't involve Cloudflare? I'm not using Cloudflare but am having the same issue with Flickr.

I get an error when testing the image proxy saying the image file...

could not be fetched or is not a valid image. The specific error message was: Unable to Connect to ssl://farm2.staticflickr.com:443. Error #0:

However, the image does load when you view it directly, for example when editing a message.

 

[DeltaHF]

There's not much you can do, unfortunately.

I did make a rather aggressive change to the image proxy's re-fetching algorithm so that it does not wait as long to retrieve broken images. At line 259 in /library/XenForo/Model/ImageProxy.php I modified the switch block like so:

(Note this is XF 1.5. It's not a plugin, it will not survive upgrades unless you go in and re-add it, and editing the source code of XF is discouraged by everyone, but I like to live dangerously sometimes.)

switch ($failCount)
{
    case 1: $delay = 0; break;
    case 2: $delay = 0; break;
    case 3: $delay = 0; break;
    case 4: $delay = 3600; break; // 1 hour
    case 5: $delay = 6 * 3600; break; // 6 hours

    default:
        $delay = ($failCount - 5) * 86400; // 1, 2, 3... days
}

So, the first three times the image fails, a reload will be attempted immediately instead of the usual, gradual back-off built into XF. It seems to have helped a bit.

 

[DeltaHF]

Sorry for the double-post (I see you already reacted to my other post, @empire, so I'm afraid you would miss this if I put it into an edit), but I forgot to mention another thing I did to help mitigate the problems I was experiencing with Flickr.

It was another pretty aggressive change: I disabled the "Image Cache Lifetime" and "Image Cache Refresh" settings (both at "0") so that all images posted on my forum are cached indefinitely.

I realize this may not be a practical option for everyone due to the hosting commitment required, but I would now recommend it to all forum owners, especially those who plan on running their sites over the long-term.

First of all, this can help with Flickr now because once your site fetches the images from their servers (or anywhere else), that is the first and last time it will ever have to do it. It won't have to do it again when the cache expires X days in the future, opening the door for more problems. Had we been doing this from the beginning, we wouldn't be dealing with any of these issues at all now.

It will also help protect you against the coming Flickr-pocalypse when Flickr starts deleting images from free accounts on February 5, 2019, which I have no doubt will remove a devastating amount of content posted in all of our forums.

I have a large (12m+ posts) forum with several image-heavy sections, and since making the change in early December, my image cache grows by roughly 2-3GB per day. I am fortunate to have a dedicated server, so I'm just going to keep adding larger hard drives to it as needed, as I expect my site will grow by about 1TB per year now.

Unfortunately, I know that won't be an option for everyone, which is why I think it is even more important for the XenForo developers to bring fully native support for cloud storage services like Amazon S3 or Backblaze B2 so it will be cheaper and more practical for everyone to host all of their own images. We all have to accept that indefinitely hosting this content is the most practical option. It is a commitment that other platforms have made (Facebook, Twitter, Reddit, etc.), and that reliability and consistency is something that users have come to expect these days.

 

[empire]

Thanks @DeltaHF – I appreciate the info. I'm a bit concerned about the Flickr-pocalypse (having barely survived the photobucket one) but the image hosting is just so expensive.