XF 2.2 How to gracefully handle unfurling errors?

[SouprMatt]

Links to certain sites do not unfurl properly. NPR is one of the most common we are seeing.

When we look at this XenForo help forum, failed unfurling links display as a regular link as a fallback. On our server however, it returns a pop-up error “Oops!" for users instead. How can we configure our server to gracefully handle a failed unfurling attempt and show as a link only? Here is an example error we receive for clarification purposes:

• ErrorException: [E_WARNING] curl_exec(): Failed outgoing connection to 'https://www.npr.org/2023/04/12/1169269161/npr-leaves-twitter-government-funded-media-label' took 6 seconds (code 28)
• src/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php:40

 

[Paul B]

Your server is returning a 500 error.

Check the server error log for the cause.

 

[SouprMatt]

Your server is returning a 500 error.

Check the server error log for the cause.

Thank you for the reply. The error I listed is from my server error log. Here is the full detail of it:

• ErrorException: [E_WARNING] curl_exec(): Failed outgoing connection to 'https://www.npr.org/2023/04/12/1169269161/npr-leaves-twitter-government-funded-media-label' took 6 seconds (code 28)
• src/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php:40
• Generated by: [removed for privacy]
• Apr 13, 2023 at 6:02 PM

Stack trace​

#0 [internal function]: XF::handlePhpError(2, '[E_WARNING] cur...', '/var/www/html/t...', 40, Array)
#1 src/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php(40): curl_exec(Resource id #71)
#2 src/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php(28): GuzzleHttp\Handler\CurlHandler->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#3 src/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php(51): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#4 src/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php(37): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#5 src/vendor/guzzlehttp/guzzle/src/Middleware.php(29): GuzzleHttp\PrepareBodyMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#6 src/vendor/guzzlehttp/guzzle/src/RedirectMiddleware.php(54): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#7 src/vendor/guzzlehttp/guzzle/src/Middleware.php(57): GuzzleHttp\RedirectMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#8 src/vendor/guzzlehttp/guzzle/src/HandlerStack.php(71): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#9 src/vendor/guzzlehttp/guzzle/src/Client.php(351): GuzzleHttp\HandlerStack->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#10 src/vendor/guzzlehttp/guzzle/src/Client.php(162): GuzzleHttp\Client->transfer(Object(GuzzleHttp\Psr7\Request), Array)
#11 src/vendor/guzzlehttp/guzzle/src/Client.php(182): GuzzleHttp\Client->requestAsync('get', Object(GuzzleHttp\Psr7\Uri), Array)
#12 src/XF/Http/Reader.php(458): GuzzleHttp\Client->request('get', 'https://www.npr...', Array)
#13 src/XF/Http/Reader.php(129): XF\Http\Reader->_request(Object(GuzzleHttp\Client), 'get', 'https://www.npr...', Array, Object(XF\Http\Stream), Array, NULL)
#14 src/XF/Http/Reader.php(81): XF\Http\Reader->requestUntrusted('get', 'https://www.npr...', Array, NULL, Array, NULL)
#15 src/XF/Http/MetadataFetcher.php(56): XF\Http\Reader->getUntrusted('https://www.npr...', Array, NULL, Array, NULL)
#16 src/XF/Service/Unfurl/Fetcher.php(22): XF\Http\MetadataFetcher->fetch('https://www.npr...', NULL)
#17 unfurl.php(71): XF\Service\Unfurl\Fetcher->fetch()
#18 {main}

Request state​

array(4) {
["url"] => string(11) "/unfurl.php"
["referrer"] => string(86) "https://thesnug.space/index.php?threads/elon-musk-closes-deal-owns-twitter.103/page-21"
["_GET"] => array(0) {
}
["_POST"] => array(4) {
["result_ids"] => array(1) {
[0] => string(4) "4670"
}
["_xfRequestUri"] => string(65) "/index.php?threads/elon-musk-closes-deal-owns-twitter.103/page-21"
["_xfWithData"] => string(1) "1"
["_xfToken"] => string(8) "********"
}
}

 

[Paul B]

That's the XF ACP error log.

I'm referring to the server error log.

 

[SouprMatt]

Is this an example of what you mean by the server log message? (I removed the IP address from the beginning of the lines)

- - [12/Apr/2023:12:36:35 -0700] "POST /unfurl.php HTTP/2.0" 500 449 "https://thesnug.space/index.php?threads/elon-musk-closes-deal-owns-twitter.103/page-21" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Safari/605.1.15"
- - [12/Apr/2023:12:37:06 -0700] "POST /proxy.php?link=https%3A%2F%2Fwww.npr.org%2F2023%2F04%2F12%2F1169269161%2Fnpr-leaves-twitter-government-funded-media-label&hash=bd698b3cb8bde835226688ab55cdfc31 HTTP/2.0" 200 279 "https://thesnug.space/index.php?threads/elon-musk-closes-deal-owns-twitter.103/page-21" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.4 Safari/605.1.15"

 

[SouprMatt]

I understand that the server is encountering an error. My question is - we have multiple threads about unfurling problems on this forum. The XenForo server for this forum seems to handle these errors gracefully by falling back to a plain link, while my web server doesn’t. I was hoping that you could shed some light on what setting(s) are used by this server when it encounters the unfurl error such that it changes it to just a plain link instead of popping up the “Oops” error. Is it as simple as increasing the curl timeout? It looks like it is 6 seconds on my server, and perhaps XenForo has a 10-second timeout where it gives up and replaces with a regular link?

 

[Paul B]

This site is the same - 6 seconds for timeout.

The same error is being returned.



That appears to be triggering a 500 error on your server so investigation will need to take place at the server level.
The error is happening outside XF.

 

[SouprMatt]

I worked with my host and they found the source of the problem. Below is a summary of what they found. I hope this information will be a good reference for anybody else experiencing similar problems. Also, perhaps the XenForo team can benefit from the knowledge that they do not have a handler for PHP errors of the E_WARNING type... by putting a handler in future versions.

What's happening is that our versions of PHP have an extra "feature"
patched in, which is to log curl_exec requests that take "too long" as a
PHP "warning", of level "E_WARNING".

The idea is to help customer find a common source of slow-performing
pages, which is that the script running the pages connects to an
external server that loads slowly, making the script seem slow when it's
not.

So far, so good: We've offered this since 2016 (see
<https://blog.tigertech.net/posts/php-updates-20160210/>), and customers
get useful info in the logs like "PHP Warning: curl_exec(): Outgoing
connection to 'http://slow.example.com/' took 16 seconds". (The warning
is separate from any subsequent PHP "E_ERROR" that curl generates about
a failure.)

But it looks like this causes problems with XenForo. XenForo has some
code that catches various problems, and in the case of curl_exec(), it
looks for and catches PHP problems of level E_ERROR from curl_exec(),
helpfully displaying the custom text seen at the bottom of that thread
("The following error occurred while fetching metadata from URL...").

But the XenForo code doesn't have a handler for the E_WARNING that's
being triggered for slow links like this, even though an E_WARNING is
"less urgent" than E_ERROR. Specifically, it looks like the code can
handle problems of either E_ERROR or E_NOTICE, but not the E_WARNING
level that's between them.

To fix this, we suppressed the E_WARNING curl_exec notices for
thesnug.space (which can be done with a php.ini option,
<https://support.tigertech.net/php-log-outgoing-connections>). And now
loading
<https://thesnug.space/index.php?threads/politicians-behaving-badly.45/page-7>
no longer shows an error.

We're sorry you experienced this problem; this is the first we've heard
of it, and we appreciate you reporting it. We will also modify our
logging patch to emit E_NOTICE rather than E_WARNING so that this
doesn't affect other XenForo users.

 

[Chromaniac]

i have been facing this issue for a while and since i am self hosted with some help from friends... i have to check if i can implement the fix offered by tigertech. but good to know that there is some documented data on this issue!

 

[Jeremy P]

Also, perhaps the XenForo team can benefit from the knowledge that they do not have a handler for PHP errors of the E_WARNING type... by putting a handler in future versions.

Just dropping a note to say XF does have handlers for PHP errors (including E_WARNING). The expectation is that anything which throws a warning should be considered a fatal error rather than having PHP try its hardest to chug along pretending everything is fine. Lower level errors (including E_NOTICE) are logged, but not considered fatal.