XF 2.2 Unfurl error where link points to domain on our server

[webbouk]

Our forum's unfurling of hyperlinks works well on links to external content, however an error has arisen since a server migration where links to any of the websites on the server do not unfurl.

Using the tool within the admin conrol panel to test the unfurling functon on specific domains returns:

The following error occurred while fetching metadata from URL https://www,maggotdrowning.com/
Could not fetch metadata from URL with error: The URL is not requestable (local: 172.27.95.135)

Our server IP address is 185.182.90.174 however we use SafeDNS provided by our hosts within a seperate control panel, but not something we've had a problem with whilst the website was on the previous server.

Is this a a DNS issue or forum related?

 

[Ozzy47]

Possibly this,

XF 2.1 - Unfurl URL - The URL is not requestable (local: 127.0.0.1)

When using the "Test URL Unfurling" tool I get: "Could not fetch metadata from URL with error: The URL is not requestable (local: 127.0.0.1)" This only occurs with my own URLs, it works fine on external URLS. External sites, such as this one, can unfurl my URLs. But I cannot unfurl my own...

xenforo.com

Or this,

XF 2.1 - Unfurl URL - The URL is not requestable (local: 127.0.0.1)

When using the "Test URL Unfurling" tool I get: "Could not fetch metadata from URL with error: The URL is not requestable (local: 127.0.0.1)" This only occurs with my own URLs, it works fine on external URLS. External sites, such as this one, can unfurl my URLs. But I cannot unfurl my own...

xenforo.com

 

[webbouk]

Thanks, that looks like the same issue - I'll look into it in the morning

 

[webbouk]

Hmmm,

I see the following in my etc/host file ...




So not sure what to edit, if anything

 

[webbouk]

It looks like the issue is connected to dns doctoring as our firewall doesn't support NAT hairpinning (loopback)

(local: 172.27.95.135) is the internal IP address of the server and Disabling DNS doctoring would break connections to the domain from the server creating issues with cPanel and cron jobs, so I've been told by our host engineer.

Is there any way around this?

@Mike @Brogan @Chris D

This is the information from our host....

The IP 172.27.95.### is the internal IP of the server. If the server wishes to connect to maggotdrowning.com it has to use the internal IP address. It cannot use the IP address on your DNS 185.182.90.174 because there is no route from the server to be able to connect from to the external IP address and therefore your server wouldn't be able to connect to maggotdrowning.com whatsoever unless the DNS result it gets is the internal IP address.

Your UKFast dedicated firewall knows this and intercepts DNS responses to your server changing 185.182.90.174 to 172.27.95.### automatically. This means if you try to connect to maggotdrowning.com from your server (such as curl requests to itself in cron jobs), everything goes through smoothly.

This feature is called DNS doctoring and is present on our dedicated Firewalls but not on servers with Shared Firewalls. This is because servers on shared firewalls have the public address set directly on the server so the DNS points to the IP set on the server so the server connects locally and no firewall is involved.

If you cannot find a way around this, our networks team can disable DNS doctoring or our Linux team could manually set the IP address for maggotdrowning.com in the hosts file however we would advise against this because it means that your server will not be able to connect to maggotdrowning.com. For example if the server needed to curl itself to trigger a cronjob or cPanel trying to validate domains on AutoSSL (LetsEncrypt).

Please get in touch with the developers to discuss this. If you let them know that the firewall does not support NAT loopback (which causes connection timeouts) and that it has DNS doctoring in place to change the IP address to the internal IP address (which causes The URL is not requestable), they should be able to provide a solution. The URL is requestable by any other application.

 

[djbaxter]

The following error occurred while fetching metadata from URL https://www,maggotdrowning.com/
Could not fetch metadata from URL with error: The URL is not requestable (local: 172.27.95.135)

I assume the comma in the URL above is a typo?

 

[webbouk]

No, I put it there to stop the link becoming active in the post

 

[Mike]

There isn't really anything we'd change regarding this. The block on requests that resolve to non-public IP ranges is to prevent a potentially significant security issue known as an SSRF. If you wanted to make changes here, it'd very likely require direct file changes.

 

[webbouk]

Thanks for the reply Mike

I think I'll leave it as it is then. To be fair it's not often we need to link to something within the website and if we do it's generally done as a formatted link with a text description.
When members post links to previous posts, especially in threads which run into several pages, it does look less confusing just with just the link rather than unfurled and using the first post of the thread as the description anyway.

Regards, Dave