Fixed Broken autolink

[kick]

Affected version

2.2.13

Broken autolink and error message: TypeError: Argument 1 passed to XF\Util\Url::urlToUtf8() must be of the type string, null given, called in /src/XF/BbCode/ProcessorAction/AutoLink.php on line 430 src/XF/Util/Url.php:50

XF\Util\Url::urlToUtf8() in src/XF/BbCode/ProcessorAction/AutoLink.php at line 430
XF\BbCode\ProcessorAction\AutoLink->unfurlLinkUrl() in src/XF/BbCode/ProcessorAction/AutoLink.php at line 149
XF\BbCode\ProcessorAction\AutoLink->XF\BbCode\ProcessorAction\{closure}()
preg_replace_callback() in src/XF/BbCode/ProcessorAction/AutoLink.php at line 146
XF\BbCode\ProcessorAction\AutoLink->filterString() in src/XF/BbCode/Processor.php at line 377
XF\BbCode\Processor->filterString() in src/XF/BbCode/Processor.php at line 360
XF\BbCode\Processor->renderString() in src/XF/BbCode/Traverser.php at line 67
XF\BbCode\Traverser->renderSubTree() in src/XF/BbCode/Traverser.php at line 39
XF\BbCode\Traverser->renderAst() in src/XF/BbCode/Traverser.php at line 22
XF\BbCode\Traverser->render() in src/XF/Service/Message/Preparer.php at line 164
XF\Service\Message\Preparer->processMessage() in src/XF/Service/Message/Preparer.php at line 129
XF\Service\Message\Preparer->prepare() in src/addons/XFMG/XF/Service/Message/Preparer.php at line 11
XFMG\XF\Service\Message\Preparer->prepare() in src/XF/Service/Post/Preparer.php at line 98
XF\Service\Post\Preparer->setMessage() in src/XF/Service/Thread/Replier.php at line 114
XF\Service\Thread\Replier->setMessage() in src/XF/Pub/Controller/Thread.php at line 441
XF\Pub\Controller\Thread->setupThreadReply() in src/XF/Pub/Controller/Thread.php at line 590
XF\Pub\Controller\Thread->actionAddReply() in src/XF/Mvc/Dispatcher.php at line 352
XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 259
XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.php at line 115
XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 57
XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2487
XF\App->run() in src/XF.php at line 524
XF::runApp() in index.php at line 20

 

[Chris D]

Is there more specific reproduction cases? It doesn’t look like we can reproduce it here.

 

[kick]

Is there more specific reproduction cases? It doesn’t look like we can reproduce it here.

Can you see the error log? You should see in the error log on xenforo.com.
This is a problem with all non-english urls.
I created a locally clean copy and checked it there and tested it on a demo version. You can check for yourself, there just the topic has a distinctive link.

Create a XenForo demo

4fda00b27df01be5.demo-xenforo.com

Thread has a distinctive character just like the example of the German language ä

 

[Chris D]

No there are no error logs that’s why I’m asking for more information.

You also didn’t explain that it was non English URLs nor did you provide any examples.

So if you have any further information then that would be appreciated.

 

[kick]

No there are no error logs that’s why I’m asking for more information.

You also didn’t explain that it was non English URLs nor did you provide any examples.

So if you have any further information then that would be appreciated.

https://4fda00b27df01be5.demo-xenforo.com/2213/index.php?threads/heutzutage-viele-hochhäuser.1/#post-1
Here is the url copied from the forum demo, if we send this link in a message, we will receive a message, as in the screenshot above. Also on the demo forum I inserted a link to the topic in the message, we see an error.



And accordingly, an error is raised for links that contain non-English words.

 

[kick]

It is somewhat related to the fact that the code has been changed, and as I understand it, the string formatter was modified, which simply checked the URL and had the following code:

if (!\XF::app()->validator('Url')->isValid($url))

Without the check:
$url = $validator->coerceValue($url);
Now it has appeared, and the attempt to convert addresses is made. So we have the following behavior:
heutzutage-viele-hochhäuser.1
When it should be transformed into the URL as follows:
heutzutage-viele-hochhaeuser.1

But we removed the URL conversion in the validation:

// romanize and deaccent to allow all accented chars to be considered valid
$value = utf8_romanize(utf8_deaccent($value));

And now, when it comes to checking the link, the value is set to "invalid." This is because a link like "heutzutage-viele-hochhäuser.1" does not pass the validation filter_var($value, FILTER_VALIDATE_URL) and return empty string. This is because FILTER_VALIDATE_URL does not support IDN in any form, meaning it does not support domains like "rødgrød.dk" or "xn--rdgrd-vuad.dk," even though the domain is active.

$validator = \XF::app()->validator('Url');
        $url = $validator->coerceValue($url);
        if (!$validator->isValid($url))
        {
            $url = null;
        }

Due to this code and the failed validation, the URL is being set to null, which is causing the error you mentioned.

 

[PaulB]

Just a few corrections so this gets solved sanely:

This is because FILTER_VALIDATE_URL does not support IDN in any form

That URL is not an IDN. It's technically an invalid URL and will be URL-encoded by the browser before being transmitted. I can see why it's not working as desired, even if the current behavior is understandable.

This is because FILTER_VALIDATE_URL does not support IDN in any form

FILTER_VALIDATE_URL works just fine with IDNs, assuming they're in punycode:

php > var_export(filter_var('https://xn--rdgrd-vuad.dk/', FILTER_VALIDATE_URL));
'https://xn--rdgrd-vuad.dk/'

FILTER_VALIDATE_URL doesn't pay any attention to reserved labels or IDNs, and it'll gladly pass along domains that are technically invalid by modern standards:

php > var_export(filter_var('https://aa--rdgrd-vuad.dk/', FILTER_VALIDATE_URL));
'https://aa--rdgrd-vuad.dk/'

However, this comes with a caveat: it isn't capable of encoding IDNs; they have to already be encoded. This aligns with the requirement for percent-encoding the path portion of the URL.

php > var_export(filter_var('https://rødgrød.dk/', FILTER_VALIDATE_URL));
false

All of this behavior is technically correct and consistent, even if it's not intuitive. Would I want to use it for a URL validator on a forum? Ehhhh... probably not. It's strict in ways that are undesirable--you probably want to encode portions of the URL, rather than rejecting them outright. It's also lenient in ways that it shouldn't be, facilitating phishing attacks like this one:

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

 

[topkurs2]

Confirm it. Same with such link for example
https://xenforo[.]com/community/

 

[lifo]

We're having the same problem.

 

[lutish]

i touch this error too

 

[duderuud]

Same error here too

 

[alexm]

And here!

If you put the subject ending in '...' https://domain.com/threads/my_subject….12345/ into a post/conversation you get an error.

• • TypeError: XF\Util\Url::urlToUtf8(): Argument #1 ($url) must be of type string, null given, called in /xxxx/xxxx/public_html/src/XF/BbCode/ProcessorAction/AutoLink.php on line 430
  • src/XF/Util/Url.php:50
  • Generated by: xxx
  • May 28, 2023 at 5:40 PM

• Stack trace​

  #0 src/XF/BbCode/ProcessorAction/AutoLink.php(430): XF\Util\Url::urlToUtf8(NULL, false)
  #1 src/XF/BbCode/ProcessorAction/AutoLink.php(149): XF\BbCode\ProcessorAction\AutoLink->unfurlLinkUrl('https://www.the...')
  #2 [internal function]: XF\BbCode\ProcessorAction\AutoLink->XF\BbCode\ProcessorAction\{closure}(Array)
  #3 src/XF/BbCode/ProcessorAction/AutoLink.php(146): preg_replace_callback('#^(?<=[^a-z0-9@...', Object(Closure), 'https://www.the...')
  #4 src/XF/BbCode/Processor.php(377): XF\BbCode\ProcessorAction\AutoLink->filterString('https://www.the...', Array, Object(XF\BbCode\Processor))
  #5 src/XF/BbCode/Processor.php(360): XF\BbCode\Processor->filterString('https://www.the...', Array)
  #6 src/XF/BbCode/Traverser.php(67): XF\BbCode\Processor->renderString('https://www.the...', Array)
  #7 src/XF/BbCode/Traverser.php(39): XF\BbCode\Traverser->renderSubTree(Array, Array)
  #8 src/XF/BbCode/Traverser.php(22): XF\BbCode\Traverser->renderAst(Array, Object(SV\SignupAbuseBlocking\XF\BbCode\RuleSet), Array)
  #9 src/XF/Service/Message/Preparer.php(164): XF\BbCode\Traverser->render('https://www.the...', Object(XF\BbCode\Parser), Object(SV\SignupAbuseBlocking\XF\BbCode\RuleSet), Array)
  #10 src/XF/Service/Message/Preparer.php(129): XF\Service\Message\Preparer->processMessage('https://www.the...')
  #11 src/addons/XFMG/XF/Service/Message/Preparer.php(11): XF\Service\Message\Preparer->prepare('https://www.the...', true)
  #12 src/XF/Service/Post/Preparer.php(98): XFMG\XF\Service\Message\Preparer->prepare('https://www.the...', true)
  #13 src/XF/Service/Thread/Replier.php(114): XF\Service\Post\Preparer->setMessage('https://www.the...', true, true)
  #14 src/XF/Pub/Controller/Thread.php(441): XF\Service\Thread\Replier->setMessage('https://www.the...')
  #15 src/XF/Pub/Controller/Thread.php(773): XF\Pub\Controller\Thread->setupThreadReply(Object(SV\SignupAbuseBlocking\XF\Entity\Thread))
  #16 src/XF/Mvc/Dispatcher.php(352): XF\Pub\Controller\Thread->actionReplyPreview(Object(XF\Mvc\ParameterBag))
  #17 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Thread', 'ReplyPreview', Object(XF\Mvc\RouteMatch), Object(XFMG\XF\Pub\Controller\Thread), NULL)
  #18 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XFMG\XF\Pub\Controller\Thread), NULL)
  #19 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
  #20 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
  #21 src/XF.php(524): XF\App->run()
  #22 index.php(20): XF::runApp('XF\\Pub\\App')
  #23 {main}
  

  Request state​

  array(4) {
  ["url"] => string(59) "/threads/can%E2%80%99t-respond-to-a-pm.262062/reply-preview"
  ["referrer"] => string(84) "https://www.domain.com/threads/can’t-respond-to-a-pm.262062/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(10) {
  ["_xfToken"] => string(8) "********"
  ["message_html"] => string(81) "<p>https://www.domain.co.uk/threads/a-huge-thank-you….255660/</p>"
  ["attachment_hash"] => string(32) "0c9028a3b7634ac5fac172f35e928836"
  ["attachment_hash_combined"] => string(88) "{"type":"post","context":{"thread_id":262062},"hash":"0c9028a3b7634ac5fac172f35e928836"}"
  ["last_date"] => string(10) "1685287602"
  ["last_known_date"] => string(10) "1685287602"
  ["load_extra"] => string(1) "1"
  ["_xfRequestUri"] => string(46) "/threads/can%E2%80%99t-respond-to-a-pm.262062/"
  ["_xfWithData"] => string(1) "1"
  ["_xfResponseType"] => string(4) "json"
  }
  }

Interestingly you don't get it on xenforo.com... but 2 sites I administer, one with hardly any mods, suffer the same thing.

 

[TPerry]

Interestingly you don't get it on xenforo.com... but 2 sites I administer, one with hardly any mods, suffer the same thing.

I see a reference to an add-on in that stack-trace.

#9 src/XF/Service/Message/Preparer.php(164): XF\BbCode\Traverser->render('https://www.the...', Object(XF\BbCode\Parser), Object(SV\SignupAbuseBlocking\XF\BbCode\RuleSet), Array)

Have you tried disabling the Sign Up abuse add-on to see if it works then? It looks like it refers to a specific ruleset?

 

[alexm]

I see a reference to an add-on in that stack-trace.

#9 src/XF/Service/Message/Preparer.php(164): XF\BbCode\Traverser->render('https://www.the...', Object(XF\BbCode\Parser), Object(SV\SignupAbuseBlocking\XF\BbCode\RuleSet), Array)

Have you tried disabling the Sign Up abuse add-on to see if it works then? It looks like it refers to a specific ruleset?

Yes, the error was in the other forum which didn't have that too.

 

[alexm]

From the other host:

Server error log

• TypeError: XF\Util\Url::urlToUtf8(): Argument #1 ($url) must be of type string, null given, called in /home/xxxx/public_html/xxxx/src/XF/BbCode/ProcessorAction/AutoLink.php on line 430
• src/XF/Util/Url.php:50
• Generated by: xxx
• May 28, 2023 at 8:58 PM

Stack trace​

#0 src/XF/BbCode/ProcessorAction/AutoLink.php(430): XF\Util\Url::urlToUtf8(NULL, false)
#1 src/XF/BbCode/ProcessorAction/AutoLink.php(149): XF\BbCode\ProcessorAction\AutoLink->unfurlLinkUrl('https://www.the...')
#2 [internal function]: XF\BbCode\ProcessorAction\AutoLink->XF\BbCode\ProcessorAction\{closure}(Array)
#3 src/XF/BbCode/ProcessorAction/AutoLink.php(146): preg_replace_callback('#^(?<=[^a-z0-9@...', Object(Closure), 'https://www.the...')
#4 src/XF/BbCode/Processor.php(377): XF\BbCode\ProcessorAction\AutoLink->filterString('https://www.the...', Array, Object(XF\BbCode\Processor))
#5 src/XF/BbCode/Processor.php(360): XF\BbCode\Processor->filterString('https://www.the...', Array)
#6 src/XF/BbCode/Traverser.php(67): XF\BbCode\Processor->renderString('https://www.the...', Array)
#7 src/XF/BbCode/Traverser.php(39): XF\BbCode\Traverser->renderSubTree(Array, Array)
#8 src/XF/BbCode/Traverser.php(22): XF\BbCode\Traverser->renderAst(Array, Object(XF\BbCode\RuleSet), Array)
#9 src/XF/Service/Message/Preparer.php(164): XF\BbCode\Traverser->render('https://www.the...', Object(XF\BbCode\Parser), Object(XF\BbCode\RuleSet), Array)
#10 src/XF/Service/Message/Preparer.php(129): XF\Service\Message\Preparer->processMessage('https://www.the...')
#11 src/addons/XFMG/XF/Service/Message/Preparer.php(11): XF\Service\Message\Preparer->prepare('https://www.the...', true)
#12 src/XF/Service/Post/Preparer.php(98): XFMG\XF\Service\Message\Preparer->prepare('https://www.the...', true)
#13 src/XF/Service/Thread/Replier.php(114): XF\Service\Post\Preparer->setMessage('https://www.the...', true, true)
#14 src/XF/Pub/Controller/Thread.php(441): XF\Service\Thread\Replier->setMessage('https://www.the...')
#15 src/XF/Pub/Controller/Thread.php(773): XF\Pub\Controller\Thread->setupThreadReply(Object(XFMG\XF\Entity\Thread))
#16 src/XF/Mvc/Dispatcher.php(352): XF\Pub\Controller\Thread->actionReplyPreview(Object(XF\Mvc\ParameterBag))
#17 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Thread', 'ReplyPreview', Object(XF\Mvc\RouteMatch), Object(XFMG\XF\Pub\Controller\Thread), NULL)
#18 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XFMG\XF\Pub\Controller\Thread), NULL)
#19 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#20 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#21 src/XF.php(524): XF\App->run()
#22 index.php(20): XF::runApp('XF\\Pub\\App')
#23 {main}

Request state​

array(4) {
["url"] => string(42) "/threads/2023-saturday.42857/reply-preview"
["referrer"] => string(56) "https://xxxx.com/threads/2023-saturday.42857/page-6"
["_GET"] => array(0) {
}
["_POST"] => array(10) {
["_xfToken"] => string(8) "********"
["message_html"] => string(81) "<p>https://xxxx.com/threads/a-huge-thank-you….255660/</p>"
["attachment_hash"] => string(32) "39628949ec09f800d1d5e0bfd0f3ddc4"
["attachment_hash_combined"] => string(87) "{"type":"post","context":{"thread_id":42857},"hash":"39628949ec09f800d1d5e0bfd0f3ddc4"}"
["last_date"] => string(10) "1685251481"
["last_known_date"] => string(10) "1685251481"
["load_extra"] => string(1) "1"
["_xfRequestUri"] => string(35) "/threads/2023-saturday.42857/page-6"
["_xfWithData"] => string(1) "1"
["_xfResponseType"] => string(4) "json"
}
}

 

[topkurs2]

Hello. This is quite serious bug (Path Disclosure).

Severity : Medium
CVSS v3.0 Score: 5.3

When will it be fixed?

 

[lifo]

Hello. This is quite serious bug (Path Disclosure).

When will it be fixed?

I am wondering same. @Chris D

 

[PaulB]

Hello. This is quite serious bug (Path Disclosure).

When will it be fixed?

This isn’t a path disclosure vulnerability. As long as you’re not in debug mode, end users won’t see the stack trace.

 

[htsumer]

Is there more specific reproduction cases? It doesn’t look like we can reproduce it here.

XF 2.2 - Error when I try to add a link

What could be the reason for this error when I try to add this link? https://kesenek.sgk.gov.tr/KesenekWeb/doc/Aylıksız_İzinli_Sayılan_Kamu_Personeli_GSS_Primleri.pdf TypeError: Argument 1 passed to XF\Util\Url::urlToUtf8() must be of the type string, null given, called in...

xenforo.com

 

[PaulB]

Here's the issue:

1. \XF\BbCode\ProcessorAction\AutoLink::unfurlLinkUrl calls \XF\Str\Formatter::prepareAutoLinkedUrl.
2. prepareAutoLinkedUrl returns an array containing url and linkText. If the URL is deemed invalid, url is null.
3. unfurlLinkUrl immediately passes the url and linkText values to \XF\Util\Url::urlToUtf8 without checking whether they're null, even though url is expected to be null at times.