We have implemented some changes in Beta 7 to address a number of concerns surrounding romanization/transliteration of strings, some are reflected in this thread, others have been somewhat of an issue for some in different scenarios over time.
Let's address the issue in this thread first.
Our previous code was based on a solution we've been using in some way or another for many years. A lot of the code was trying to solve problems and edge cases that at one point didn't have a reasonable native solution. We unraveled a few of the more redundant bits of that code in Beta 6 but for the most part the behaviours were generally unchanged from before.
Specifically code surrounding romanization which is often used in URLs behaved largely the same as it already had done, and it turns out that the approaches were fairly opinionated.
For example
ö and
ü were transliterated to
oe and
ue. This is not incorrect necessarily, but it depends on your locale. In German and potentially some other languages it is correct, but for Turkish and some others it should be
o and
u.
We took a little bit of inspiration here from the amazing work produced by the Symfony team, specifically their String component and we have entirely overhauled the process for both normalizing and transliterating strings.
These changes will only apply if you have the intl extension available.
What does this mean in practice? Well, as you can see from the new title I've given this thread, the characters
ö and
ü are now transliterated to
o and
u.
But what about German language forums? Well, the specific rules we use for transliteration are now locale based (based on the default language of the forum).
If our default language on this forum was German, the characters
ö and
ü would be transliterated to
oe and
ue.
So, with that, I think we can safely call this fixed. But there's more!
On a semi-related note, over the years customers have sometimes expressed a concern over the appearance of emoji in URLs. There's nothing invalid about emojis being in URLs. Behind the scenes they are URL encoded, and most browsers will display them as the correct emoji icon. This behaviour remains unchanged unless "Romanize titles in URLs" is enabled. In which case, we now have this new option to control how emojis appear in URLs:
By default, we will now convert the emoji into a string based on the emoji name. You may also decide to keep the emoji, something that previously wasn't possible when romanization was enabled. Or you may decide to remove the emoji entirely. You can see examples of these in the screenshot above.
And the end result can be seen in the URL of this thread:
Code:
https://xenforo.com/community/threads/bug-with-turkish-characters-such-as-o-and-u.221332/
The
character is converted to the word
bug. And
ö and
ü are converted to
o and
u.
. I'm glad you finally found a permanent solution to this problem.
