XF 2.2 Image Proxy tries to proxy attachments

[Fullmental]

I have a weird issue going on. When I backfill my production site to a dev environment, attachment links that contain a full URL to a media item break. For example, if my site is forum.com and I have an attachment /media/funny-gif.57 that gets attached to a post via the URL options from the media page, it works on the live site with the IMG source URL as [img]https://forum.com/media/funny-gif.57/full[/img]. But when I have my dev site running on a localhost environment, Xenforo tries to proxy this image URL by attempting to fetch it from forum.com, even though it exists and is accessible on localhost through its own canonical URL structure, simply because the live site's domain was used in the embed by an end user.

OK, no problem. I went into the image proxy settings in ACP and added my primary domain to the bypass list. All should be good, right? Wrong, the proxy process doesn't seem to acknowledge and bypass these types of routed URLs, it seems to only work when the link is a direct image URL, not a URL that is an attachment route.

Short of running a SQL script to replace every instance of my live site URL in all messages with an equivalent localhost path, how else can I get Xenforo to stop trying to proxy attachments from my own site when backfilling to a dev instance? I do not want to disable image proxying completely because that would defeat the purpose of testing on a dev site - it should match the live site as closely as possible and that includes proxying behavior, especially if I'm making changes to how the proxy works or testing bypass patterns or any other options. That said if things are breaking in unexpected ways before I do anything, how am I to know if a change I made caused something else to break or not?

 

[Fullmental]

Any help on this one? Is this perhaps a bug or "feature" due to the way the URL is configured? I wonder if the bypass isn't working because there is technically no image extension in these URLs they are grabbing from the "copy image link" section of the media gallery.

Here's a complete example. I have this test image:



If I click the "copy image link" button to get me the full image link URL, and paste it in image bbcode tags (or use the "copy image bb code" option directly, this works and displays normally on the production site:



If I then take a backfill to our test site, it oddly enough renders the image in the quick reply visual editor section:


But as soon as I preview or click "post", the image breaks:



Clicking the broken image link brings me to the proxy.php? link for that image, even when I have our own domain entered in the bypass image proxy list:


This also doesn't seem like appropriate behavior as it is, because why would the image fetch and render properly in the visual editor, then break on post/preview? This all just seems wrong.

 

[Jeremy P]

For example, if my site is forum.com and I have an attachment /media/funny-gif.57 that gets attached to a post via the URL options from the media page, it works on the live site with the IMG source URL as [img]https://forum.com/media/funny-gif.57/full[/img].

FWIW an attachment embedded via [img] is not treated any differently than any other hot-linked image (ie. from a third-party site). I imagine it's proxied because it's embedding what is essentially a third-party image from your live installation on your development environment.

OK, no problem. I went into the image proxy settings in ACP and added my primary domain to the bypass list. All should be good, right? Wrong, the proxy process doesn't seem to acknowledge and bypass these types of routed URLs, it seems to only work when the link is a direct image URL, not a URL that is an attachment route.

I can't reproduce this, and per above there shouldn't be any distinction. If I exempt a live domain by placing example.com in the bypass list then any [img] from that domain are not proxied.

This also doesn't seem like appropriate behavior as it is, because why would the image fetch and render properly in the visual editor, then break on post/preview? This all just seems wrong.

I can't really account for why the bypass wouldn't work as it looks to be configured fine, but the visual editor has never run images through the proxy.

 

[Fullmental]

OK. I think I see what's going on here. Our media gallery image permissions are set to disallow guest viewing. When I am on my test site, I'm using the same browser session that is logged into my production site, so it appears that, when the proxy is disabled the request coming from my browser seems to be treating my request as being from a "logged in" user, perhaps due to an active session token? If I try accessing the same page on a different browser or when logged out, the image breaks even when the proxy is disabled. Enabling the image proxy obviously has the server itself try to fetch the image, no session token involved so it fails to fetch it with a 403 error. This would also explain the behavior on the visual editor because if it's not fetching via proxy then it uses the same session token to successfully fetch the image when I am editing the post.

I never would have expected this behavior from the browser, but this appears to be why you would be unable to recreate it either? Especially if the media gallery is configured to allow guest viewing by default (I can't remember if we've changed that setting or not ourselves). But what that doesn't explain is why the proxy bypass doesn't seem to be working in this regard? If what I've guessed here is accurate, a bypass should direct the browser to render the image directly, but I still get a proxy.php? link if I right click and "view image in new tab" on my browser. Shouldn't that image link be the direct link that would either render the image (if logged in) or direct me to the login page on my main site?

 

[Fullmental]

To add to the confusion here...I attempted to try an image that is directly hosted without the media gallery at all. Using this image, I still get a failed proxy attempt



The actual link here should be globally viewable as it's used in our banner ad system and the directory is managed server-side, not through the xenforo permissions system, indeed I can click this link and open in a private window or other browser and it works: https://nuzlockeforums.com/pics/user_submissions/09_23/rubombee_notimelikethepresent.png

So...that takes the media gallery and view permissions from XF itself out of the equation entirely. I don't understand why we would be receiving a 403 error here through the proxy test when it's perfectly accessible everywhere else?

 

[Jeremy P]

But what that doesn't explain is why the proxy bypass doesn't seem to be working in this regard? If what I've guessed here is accurate, a bypass should direct the browser to render the image directly, but I still get a proxy.php? link if I right click and "view image in new tab" on my browser. Shouldn't that image link be the direct link that would either render the image (if logged in) or direct me to the login page on my main site?

Yes. I'm not sure either but I can't reproduce that behavior myself. If a domain is bypassed then the link is direct.

So...that takes the media gallery and view permissions from XF itself out of the equation entirely. I don't understand why we would be receiving a 403 error here through the proxy test when it's perfectly accessible everywhere else?

No idea but if it's served directly it's likely down to web server configuration or a WAF (Cloudflare, etc.). For what it's worth I don't have any with this locally either:

 

[Fullmental]

OK, so I have solved half of the problem. The link above, turns out we had an external proxy server configured in the config.php which was not being commented out properly for our test site. This was causing a double proxy issue. That problem solved with commenting out the config line there.

Media bbcode, still a problem. Again, with the bypass active for the domain our site is still inexplicably trying to proxy it. I got it to work by toggling development mode and debug mode off and on again and it worked temporarily for about 30 minutes before it broke again. That's as close as I got.

Unless anyone has an idea on this one, I'm probably going to have to give up on it. Since it's just a test instance, I guess we'll deal with working around it when the use case does come up.