Hosted images broken since upgrade patch

[Mr Lucky]

Images are showing correctly on my test forum (before upgrade patch), however on my live forum (with patch) they show a broken image symbol.

These are images that are hosted on my server.

However the links are good because if I click on edit, they show up in the editor. They also work if I untick Proxy Image, however this is not an option as mentioned in the ACP: Proxying of images is especially important if you are running your site through SSL (HTTPS).

 

[Mike]

What's the URL to the images?

As part of the patch, redirects are no longer followed and this is the likely cause. Unfortunately, this is a very important part of the patch.

 

[Mr Lucky]

What's the URL to the images?

As part of the patch, redirects are no longer followed and this is the likely cause. Unfortunately, this is a very important part of the patch.

The images (at least the ones I know of that are broken) are at http://cafesaxophone.com/x-pic/..."



I recently switched to SSL, so it is important that this works as mentioned in the ACP:

Proxying of images is especially important if you are running your site through SSL (HTTPS).

 

[Mike]

Yeah, those are redirecting from http to https so the failure is generally expected (unfortunately).

It would be an option to replace http://cafesaxophone.com with https://cafesaxophone.com in existing messages and that would sort the common case.

 

[Mr Lucky]

This upgrade is really screwing my forum.


For example:


https://cafesaxophone.com/threads/grassi-saxophones.13245/page-5#post-302491

shows broken image links.

Yet the links are fine when you follow the actual link outside of xenforo:

https://s14.postimg.org/ugd98aw0h/Photo_on_9_12_16_at_2_20_PM.jpg


How can I fix this?

 

[Chris D]

This is basically what Mike said above. Notice the URL you posted:

https://s14.postimg.org/ugd98aw0h/Photo_on_9_12_16_at_2_20_PM.jpg

vs the URL you are redirected to:

https://s14.postimg.io/ugd98aw0h/Photo_on_9_12_16_at_2_20_PM.jpg

So if I post the first URL:

We'll reject it because it performs a redirection, and a URL that redirects can leave you susceptible to the SSRF exploit.

If I post the second URL:

It will work fine.

The solution is to identify broken images (the Image Proxy Log in the Admin CP may help with this) and use something such as the Post Content Find and Replace add-on to fix things, if necessary.

 

[Mr Lucky]

The solution is to identify broken images (the Image Proxy Log in the Admin CP may help with this) and use something such as the Post Content Find and Replace add-on to fix things, if necessary.

I'm trying to use that, but seems very complicated. Are you saying all images that were .org are now .io

????

It was all working nicely before the upgrade patch.

 

[Chris D]

It was also a potential attack vector that, although not easy to exploit, if it ever was, that'd be far more inconvenient

I'm trying to use that, but seems very complicated.

I've provided some assistance in that thread.

Are you saying all images that were .org are now .io

Yes, exactly.

 

[Mr Lucky]

Are you saying all images that were .org are now .io

Yes, exactly.

Doesn't seem to be:

This is .org:

http://shoutsistershout.org/pic/laurie.jpg

But if I replace .org with .io it doesn't work:


http://shoutsistershout.io/pic/laurie.jpg

 

[Pierce]

I assume trying to add code to check the curl response, and loop the process up to 5 times and if its still giving back 301's, reject, if it stops and validates as a correct url @ 200 download the image at the url found, is a bit heavy server side? or just really ugly?

Or there is another vector of attack to that approach which I am not thinking of.

What about XF2 has this been accounted for?

 

[Chris D]

Doesn't seem to be:

This is .org:

http://shoutsistershout.org/pic/laurie.jpg

But if I replace .org with .io it doesn't work:


http://shoutsistershout.io/pic/laurie.jpg

You've generalised that a bit too much. I was specifically referring to URLs containing postimg.org now being postimg.io.

I can't actually see a problem with this image:

 

[Chris D]

I assume trying to add code to check the curl response, and loop the process up to 5 times and if its still giving back 301's, reject, if it stops and validates as a correct url @ 200 download the image at the url found, is a bit heavy server side? or just really ugly?

We already have/had protection against redirect loops and requests that take too long.

This particular situation is to prevent a potential SSRF (server side request forgery).

Put simply, a publicly accessible URL if accessed by your server (such as via the image proxy system) could issue a 301 redirect to a local resource/service on the server. For example, if a redirect was issued to http://localhost:3306 and the server followed that, it could be a way to bypass the firewall and remotely access MySQL. This example is a bit extreme and unlikely, but you get the idea.

This is why redirects are now blocked.

 

[Mr Lucky]

You've generalised that a bit too much. I was specifically referring to URLs containing postimg.org now being postimg.io.

I can't actually see a problem with this image:

That one is fine, it was when replacing .org with .io that there was a problem.

You implied all .org should be .io

If it's not all of them, how do I know which ones to replace?

I'm very confused now as it was all working fine with proxy Image ticked as advised, now it isn't and I'm not sure how to fix it. I can fix links to me own URL, but how do I identify all the links to external URLs that have now stopped working?

 

[Pierce]

@Chris D I understand the issue.

I see in the source there's checks for local IPS ipv4 and ipv6.

Instead of following the redirects blindly (which is what is happening when you send it off to Zend, and you have maxredirects @ 0 now), but query only for the headers only(the headers that create the 301).

So the flow would be:

Step1:

query header for example.org/image.jpg

which the header will tell you that it 301 directs to: example.org/redirect_image.jpg

Step 2

Query header information for example.org/redirect_image.jpg

which continues to 301 redirect and you keep downloading the headers until you hit a 200(successful response) and then and only then download the image or you run out of loops(5 usually) or you find a localhost or any private ip adress and it fails for security reasons...

 

[Chris D]

You implied all .org should be .io

That wasn't my intention. We were talking specifically about an image that was on a certain host postimg.org and their URL has now changed to postimg.io.

If you use the post content add-on, you can change all postimg.org URLs to postimg.io.

I honestly doubt there will be many other examples of broken images so as long as you resolve the most obvious issues (e.g. those hosted on your own domain, and this postimg.org URL change) there's unlikely to be many (if any) more.

 

[Mr Lucky]

That wasn't my intention. We were talking specifically about an image that was on a certain host postimg.org and their URL has now changed to postimg.io.

Aha, I misunderstood, I though you meant change all .org to .io. I must admit that seemed a bit odd.


https://xenforo.com/community/threads/hotspots-by-waindigo-deleted.54028/#post-576878

That wasn't my intention. We were talking specifically about an image that was on a certain host postimg.org and their URL has now changed to postimg.io.

If you use the post content add-on, you can change all postimg.org URLs to postimg.io.

I honestly doubt there will be many other examples of broken images so as long as you resolve the most obvious issues (e.g. those hosted on your own domain, and this postimg.org URL change) there's unlikely to be many (if any) more.

OK, thanks I will do that.

BTW, I spotted a broken image on xenforo.com, is this due to the same issue?

https://xenforo.com/community/threads/hotspots-by-waindigo-deleted.54028/#post-576878

 

[Chris D]

Actually it is, yes. We'll look into that.

 

[Mr Lucky]

Actually it is, yes. We'll look into that.

Happy to be of service

 

[Mr Lucky]

Is there a new patch to fix this or is it only the whole new version 1.5.11?
I am asking because one forum I administer has not yet had its licence upgrade and is currently on a patched 1.5.7

I am unable to contact the owner to get him to upgrade the licence (which he intends to do but can be hard to contact) I wanted to use my spare time today to convert this forum to SSL, but it the patch still causes this issue that won't be an option.

 

[Mike]

The only way to re-add redirect support is to upgrade to 1.5.11.