XF 2.1 Emojis are converted to character entity syntax when posted

[Christer L]

On my recently launched forum (migrated from vBulletin 3.8) emojis entered using the XF post editor are converted to hex codes when I preview/post it.
Is there some option I need to set or what is going on?

The collation in my database is utf8mb4_swedish_ci, not the utf8mb4_general_ci thay XenForo installs as - could that be the problem?
Emojis enter by the keyboard (OSX Emoji & Symbols) work fine - so the collation/charset seems to allows emojis at least.

Example on how it looks. The second example is the Swedish flag added from editor, and then from my keyboard.

 

[Chromaniac]

i guess check the admin.php?smilies/ section. migration from vB might have created a mess in this section and you might want to consider restoring it to xenforo default settings?

 

[Christer L]

Thanks. But after more test, it turns out the problem is with the emoijis, not the smilies - smiles work fine, but emojis entered using editor are shown as &#... I have edited the thread title to reflect this.

 

[Paul B]

Have you carried out all of the steps detailed here: https://xenforo.com/xf2-docs/manual/unicode/

 

[Christer L]

Have you carried out all of the steps detailed here: https://xenforo.com/xf2-docs/manual/unicode/

Yes, this was a fresh install with fullUnicode from the start.

But I then changed the collation to utf8mb4_swedish_ci though, as I want that instead of the general_ci that is hardcoded into XenForo - to get proper sorting and searching for Swedish. (Unfortunately I did not check emoji support before changing).

As emojis are shown when entered from my keyboard, (both OSX and my iPhone) but not when added from editor, it seems the collation is fine - it can actually store emojis.

So to me the problem seems more related to the editor, something is "turning off" emoji support, showing hex character entity syntax instead. Maybe it thinks there is no full unicode support because I changed the collation to utf8mb4_swedish_ci?

I have looked for code in XF to see if there was a check "if utf8mb4_general_ci show emoji, else show &&#..." or something but not found any.,

 

[Christer L]

UPDATE: this problem was caused by not having PHP:s setting "default_charset" set to "utf-8'
It was still on "iso-8859-1" which we hade before converting to XenForo.
That caused html_entity_decode() to fail in EmojiFormatter; it should have converted the character entity to native character, but it didn't

(Maybe XenForo install should check that the default PHP charset really is set to "utf-8" before proceeding, or the XenForo requirements script detect this)

Case closed, something new learned