Implemented True UTF-8 support - MySQL utf8mb4

[Xon]

MySQL's utf8 data type only supports 3-byte characters, not the full 4 byte characters.

MySQL only recently as of MySQL 5.5 now supports the full UTF-8 encoding set via the table coolation type utf8mb4.

This means ios emoji icons (and likely other mobile devices who move to offer the same support) and non-English characters are impacted.

After derailing a support ticket, and seeing it happening to another one;

• https://xenforo.com/community/threads/some-unicode-characters-not-recognized.89945/
• https://xenforo.com/community/threads/emoji-icons-posting-from-ios-devices-to-xenforo.25722/

People involved in those last threads who may be interested: @RobinHood, @Tracy Perry, @cclaerhout, @Omar Bazavilvazo @tyteen4a03

 

[TPerry]

Not so much having the space as it is having 3rd party code back up something that should be done from the CLI to begin with.

Agree with add-on process generally. I use system CRON jobs, but I thought you could create your own CRON job in XenForo to perform the same function with a custom PHP instead of a script.

 

[gfc]

Anyone for doing an addon for this?

 

[Mert]

I sincerely hope XF 2.0 will have full UTF8 support and a decent conversion tool designed to run on CLI

 

[TeflonDon]

Hi there,

I've done a simple addon translating the Input to the EmojiOne format, and then converting it back into smileys at render time ... Works well, without any impact on the database. Just saying ...

Is this publicly available?

 

[Kintaro]

Hi there,

I've done a simple addon translating the Input to the EmojiOne format, and then converting it back into smileys at render time ... Works well, without any impact on the database. Just saying ...

Is this downloadable from the RM?

 

[Fred.]

If this is not supported in XF2 I need to find a solution for this.
I hope it is, otherwise I need an add-on for that.

 

[Valter]

Any news on this? It's becoming must-have for mobile users.

 

[Brent W]

Any news on this? It's becoming must-have for mobile users.

https://xf2demo.xenforo.com/threads/whats-new-in-xenforo-2.114/

Emoji (utf8mb4) support
If you run a version of MySQL that supports utf8mb4, you can convert your tables (through a script) to use that to allow emojis to be used in posts: ️️ Currently, the underlying emojis come from the OS/browser so they may differ between visitors.

 

[Arty]

Any news on this? It's becoming must-have for mobile users.

It works in XF2. See https://xf2demo.xenforo.com/threads/i-wonder-🦃.190/

 

[king8084]

@Daniel Hood make this possible for xf 1.0

 

[Brent W]

@Daniel Hood make this possible for xf 1.0

Good luck with that...

 

[king8084]

Good luck with that...

maaaaaan rewriting all my mods just so i can have emoji

this gonna be tough choice

 

[king8084]

i am going to attempt doing this tomorrow, pray for my soul @Chris D

 

[Chris D]

I'll just advise you not to do it.

 

[Fred.]

I also think it's too risky to do, It's so easy to mess things up.
It's not too hard to make it work, but the problem is when things change. When you have to update your forum or add-ons. When you forget to update 1 line or when you don't do it in the correct order you're ******

 

[Chris D]

It's not just database changes required but also changes to the core code to ensure that full Unicode isn't stripped using the input filterer.

 

[king8084]

I'll just advise you not to do it.

it didnt work, i had to roll the forum bacxk

 

[Chris D]

It's almost as if I knew what I was talking about when I advised you not to do it

 

[king8084]

It's almost as if I knew what I was talking about when I advised you not to do it

reading op and other thread i thought it would be possible

going 2.0 is not an option for me right now and everyone wants emojis

meh

 

[Chris D]

Well, something will have to give way at some point. Obviously no one can go to 2.0 "right now" as it's still an Alpha. It will be released this year but now is the time to start planning how you're going to make the move, what work is required, how much it will cost (assuming development is involved) etc. That will at least allow you to tell everyone who wants emojis what the ETA is for that.