XF 1.5 Untrusted Http Client

Sunka · Feb 23, 2016

A new config.php option $config['untrustedHttpClient'] has been added to allow you to specify a custom configuration when making untrusted HTTP calls such as via the image proxy. For example, this can allow you to direct the connection through a proxy to keep your server's IP hidden.

Regarding this in newest xenforo version, could someone give some examples and elaborate this option little more.
My forum is on https and I am running xenforo proxy for images.

Thank you

dieg0 · Feb 14, 2017

Mike said:
No that wouldn't directly be an option. I suppose technically you could run a local proxy that you'd connect to which itself would then work through multiple other proxies as needed, though I'm not sure if there is any proxy package that provides that.

I'm using this nginx config as a proxy on a different server:

server {
listen 8080;
location / {
resolver 8.8.8.8;
proxy_pass $scheme://$http_host$request_uri;
}
}

It isn't working for HTTPS images.

I have also tried a adding the next line of code to the XF proxy adapter with no luck

stream_context_set_option($this->socket, 'ssl', 'peer_name', $host);

I'm running PHP 5.4.16 and xf 1.5.3 patched.

The error that it's throwing :

https://i.imgur.com/Y8mRMqW.png could not be fetched or is not a valid image. The specific error message was: Unable to connect to HTTPS proxy. Server response: HTTP/1.1 400 Bad Request Server: nginx Date: Tue, 14 Feb 2017 20:32:18 GMT Content-Type: text/html Content-Length: 166 Connection: close

Also, my xenforo config file looks like this:

$config['untrustedHttpClient'] = array(
'adapter' => 'Zend_Http_Client_Adapter_Proxy',
'proxy_host' => '8x.xx.xx.xx',
'proxy_port' => 8080
);

Any ideas?

eva2000 · Feb 15, 2017

@dieg0 you need a forward proxy like tinyproxy, squid, 3proxy not a reverse proxy nginx etc

forward proxy hides client ip (in this case the xenforo host server is the client pulling remote images)
reverse proxy hides server ip

dieg0 · Feb 15, 2017

Thanks @eva2000 !

I just set up Squid and it seems to work... most of the times. There are certain https images that refuse to load.

This one works:
https://i.imgur.com/Nw2mgYc.jpg

However these do not:
https://www.seedprod.com/wp-content/uploads/2014/08/2014-08-10_09-32-24-600x348.png could not be fetched or is not a valid image. The specific error message was: Received unexpected response code 403 (Forbidden)
https://cs.stanford.edu/people/karpathy/randomForestSpiral.png could not be fetched or is not a valid image. The specific error message was: Received unexpected response code 400 (Bad Request)

Tested both of them with untrustedHttpClient disabled and they work just fine

dieg0 · Feb 16, 2017

dieg0 said:
Thanks @eva2000 !

I just set up Squid and it seems to work... most of the times. There are certain https images that refuse to load.

This one works:
https://i.imgur.com/Nw2mgYc.jpg

However these do not:
https://www.seedprod.com/wp-content/uploads/2014/08/2014-08-10_09-32-24-600x348.png could not be fetched or is not a valid image. The specific error message was: Received unexpected response code 403 (Forbidden)
https://cs.stanford.edu/people/karpathy/randomForestSpiral.png could not be fetched or is not a valid image. The specific error message was: Received unexpected response code 400 (Bad Request)

Tested both of them with untrustedHttpClient disabled and they work just fine

@Mike any ideas?

Just tried to wget the seedprod.com image from the server squid is installed on and got a 200 OK server response. Why the 403 forbidden from xenforo?

dieg0 · Feb 17, 2017

This is the squid setup guide I followed: https://hostpresto.com/community/tutorials/how-to-install-and-configure-squid-proxy-on-centos-7/

up until the "Using Basic Authentication with Squid" part which I believe we don't need.

I think it would be great to have some standard Squid setup/configuration guide on this forum in case someone else decides to use the UntrustedHttpClient option and has no idea about proxies.

Mike · Feb 17, 2017

dieg0 said:
@Mike any ideas?

Just tried to wget the seedprod.com image from the server squid is installed on and got a 200 OK server response. Why the 403 forbidden from xenforo?

I'm not totally sure. If you submit a ticket with FTP details, I can probably at least get a fuller error message out.

dieg0 · Feb 18, 2017

Mike said:
I'm not totally sure. If you submit a ticket with FTP details, I can probably at least get a fuller error message out.

Can you confirm the UntrustedHttpClient setting works on xenForo 1.5.3?

I'm just asking because it looks like it was added with the release of 1.5.6

https://xenforo.com/community/threads/xenforo-1-5-6-released.112942/

Mike · Feb 18, 2017

It was added in 1.5.6, so no it doesn't work there. (We will generally assume you're running the latest version unless you mention otherwise. There are also other changes beyond this option that may be relevant.)

dieg0 · Feb 18, 2017

Mike said:
It was added in 1.5.6, so no it doesn't work there. (We will generally assume you're running the latest version unless you mention otherwise. There are also other changes beyond this option that may be relevant.)

Just upgraded to 1.5.12. Same issue.

The thing is it works fine from my forum's server:

curl -I https://www.seedprod.com/wp-content/uploads/2014/08/2014-08-10_09-32-24-600x348.png --proxy MYtinyproxyBOX:8080
HTTP/1.0 200 Connection established
Proxy-agent: tinyproxy/1.8.3

HTTP/1.1 200 OK
Date: Sat, 18 Feb 2017 16:02:19 GMT
Content-Type: image/png
Content-Length: 105701
Connection: keep-alive
Set-Cookie: __cfduid=d8e33367aac8e18918c55c8231b1cc80b1487433739; expires=Sun, 18-Feb-18 16:02:19 GMT; path=/; domain=.seedprod.com; HttpOnly
Last-Modified: Wed, 03 Aug 2016 20:54:04 GMT
ETag: "57a259ec-19ce5"
CF-Cache-Status: HIT
Expires: Mon, 06 Mar 2017 16:02:19 GMT
Cache-Control: public, max-age=1382400
Accept-Ranges: bytes
Server: cloudflare-nginx
CF-RAY: 3332b3e65e7915e3-FRA

However from the admin panel of xf 1.5.12 admin.php?tools/test-image-proxy I get this error:

https://www.seedprod.com/wp-content/uploads/2014/08/2014-08-10_09-32-24-600x348.png could not be fetched or is not a valid image. The specific error message was: Received unexpected response code 403 (Forbidden)

I'm running PHP 5.4.16

Mike · Feb 20, 2017

I'm pretty sure that you're issue is one that we just traced through recently.

The latest example uses CloudFlare which potentially requires SNI support. PHP has a bug with how SNI is configured (when using a proxy) that appears to be changed in PHP 5.6. I believe upgrading to PHP 5.6 will resolve this issue for you. (I don't believe we will be able to workaround this beyond that.)

dieg0 · Feb 20, 2017

Mike said:
I'm pretty sure that you're issue is one that we just traced through recently.

The latest example uses CloudFlare which potentially requires SNI support. PHP has a bug with how SNI is configured (when using a proxy) that appears to be changed in PHP 5.6. I believe upgrading to PHP 5.6 will resolve this issue for you. (I don't believe we will be able to workaround this beyond that.)

Our entire infrastructure runs on CentOS 7 and PHP 5.6 is not in their repos yet. I know I could get it from somewhere else but then we would need to update a bunch of stuff also. It would be a mess.

Just upgraded to php-fpm-5.4.16-42.el7.x86_64

Isn't there a workaround sort of like this one?

Code:

if (PHP_VERSION_ID >= 50600)
        {
            stream_context_set_option($this->socket, 'ssl', 'peer_name', $host);
        }

Mike · Feb 20, 2017

You can install the Remi repos to get access to more recent PHP versions. There's some example discussion here: https://www.digitalocean.com/community/questions/how-to-install-php-5-6-on-centos-7-0-x64

But otherwise, the direct alternative to that code doesn't work in PHP 5.4 unfortunately in this case.

dieg0 · Feb 21, 2017

php 5.6 fixed it.

dehness · Apr 11, 2018

Mike said:
As an example of that configuration:

Code:

$config['untrustedHttpClient'] = array( 'adapter' => 'Zend_Http_Client_Adapter_Proxy', 'proxy_host' => '123.123.123.123', 'proxy_port' => 80 );

You'll need an HTTP proxy to access (either one you run on a different server or an external one).

Hi Mike, is this still the correct syntax with XF2? I added it to my config.php but it seems to have no effect.

Chris D · Apr 11, 2018

Slightly different in XF2.

PHP:

$config['http']['proxy'] = 'tcp://123.123.123.123:80';

dehness · Apr 11, 2018

Chris D said:
Slightly different in XF2.

PHP:

$config['http']['proxy'] = 'tcp://123.123.123.123:80';

Thanks Chris, worked like a charm.

dieg0 · Jul 3, 2019

Hi guys,

Just switched from centos to ubuntu and this is the error message I get from using

$config['http']['proxy'] = 'tcp://123.123.123.123:8080';

cURL error 7: Unsupported proxy scheme for 'tcp://123.123.123.123:8080' (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)

Any Ideas?

Chris D · Jul 3, 2019

Try changing tcp:// to http://.

dieg0 · Jul 3, 2019

Chris D said:
Try changing tcp:// to http://.

Works like a charm.

Thank you Chris.

DragonByte Tech · Jul 3, 2019

This is only tangentially related to the topic at hand, but this thread taught me what the purpose of the untrusted client is

Time to dig through my HTTP calls to see which ones I need to convert

XF 1.5 Untrusted Http Client

Well-known member

Active member

Well-known member

Active member

Active member

Active member

XenForo developer

Active member

XenForo developer

Active member

XenForo developer

Active member

XenForo developer

Active member

Well-known member

XenForo developer

Well-known member

Active member

XenForo developer

Active member

Well-known member

Similar threads

We value your privacy