XF 2.4 Tiptap: A new editor for XenForo

2.4 editor flare.webp
Rich text editors are hard. When you drill down into how they actually work, it's a wonder why there are so many of them, and why developers would willingly spend a significant part of their life working on one. At XenForo we have been through ... counts on fingers ... three different rich text editors in our history, so if we're being generous we can at least attest to what a feat of engineering they are and how difficult they can be to get right -- and all we have to do is implement them.

That's no easy feat either, though. We have an increasing number of features that we need from an editor, we have a layer of conversion from HTML to BB code (and back again), and we have an overall user base of millions of members across all of our customers who all seem intent on breaking our editor in weird and wonderful ways. So picking a new editor is fraught with risk and challenges. Sometimes, even when we pick the right editor, it doesn't always stay that way.

Over recent years we've been keeping a close eye on the developments within the rich text editor space and while there are a few interesting projects, there is only one very clear winner, and that is Tiptap.

What is Tiptap?​

Tiptap is a completely modular, open source and headless editor framework. Headless means that there is no provided user interface. Modular means you can enable only the functionality that you need. It is so modular that, technically, you don't even have to enable paragraphs or text (though that might impede its usefulness). It also has a fantastic extension system that allows us to not only create our own extensions, but also extend existing extensions! (more on that later).

Tiptap, which is itself powered by the amazing ProseMirror toolkit, does not simply attempt to decorate input with arbitrary HTML like most rich text editors. Rather, it has a very strict (but extendable) schema made up of Marks and Nodes. Marks are inline formatting (such as bold and italic) and Nodes are block-level formatting (such as paragraphs, quotes, and code blocks).

Your first look​

Strap in, because there's lots to show you. But first, an artist's representation of how it might look in either light or dark styles:

2.4 editor gradient.webp


The overall look and feel of the editor isn't 100% finalised at this point, but the current iteration, as pictured above, does lean into being familiar. One of the notable changes currently is that we have moved the toolbar to the bottom of the editor, making the controls more accessible, particularly on smartphones.

Want it at the top? No problem, that's a single line of CSS to make that happen.

The toolbar is, as ever, managed entirely through the existing editor button manager in the admin control panel, so the toolbar itself can be made as full-featured or as minimalist as you like.

Great for users and power users alike​

For a lot of things, you might not even need to use the toolbar. The brave amongst you might even choose to disable the toolbar entirely, thanks to the simplicity with which many editor features can be used.

As part of the overall schema that powers Tiptap, "input rules" can be defined by developers for each extension, which, in short, are "Markdown-style" editing:



Power users will not only be able to use Markdown-style syntax to produce content - which is enabled for a significant amount of editor functionality - but they will also be able to see the formatting change in real-time.

What you see is really what you get​

In fact, all core editor functions have a visual representation of their content, whereas they might not have previously. We are putting an end to seeing raw BB code syntax in the editor (unless you type it yourself, which still works):



This level of visual clarity for what your final content will look like extends even to code blocks, which are now fully syntax highlighted in the editor. The syntax highlighting is provided by highlight.js. As a result, we have upgraded our BB code syntax highlighting for code blocks to also use highlight.js going forward. We are including more languages (and more relevant languages) by default, and it remains easy to add additional ones to fit your particular niche.



It truly is a "what you see is what you get" experience, and to underline that even more, wait until you see what happens when you paste an embeddable link (best viewed full screen):



A consistent UI powered by XenForo​

As we mentioned earlier, Tiptap is a completely headless editor, meaning we have built the entirety of the UI that you can see. This is now more consistent and more in line with the default XenForo styling than ever. With our previous editors, there was always a hybrid of different UI pieces: some from the editor itself, others that we bolted on. For example, in our current editor implementation there are three distinct UI components we use - Froala pop-ups (insert link), overlays (insert media) and menus (emoji). With the new editor, everything you see is crafted from default XenForo functionality and styling, ensuring consistency:
CleanShot 2024-12-17 at 16.36.31.gif


By the way, the "Insert image" menu of the editor now supports multiple file uploads, including when dragging and dropping!

An exciting experimental editing experience​

While the new UI for the new editor will be very familiar to most, because we now have full control over what the editor can do, we are also experimenting with more innovative methods for writing content. Let's have a sneak peek now at what that might look like (best viewed full screen):



We haven't yet fully decided what form this new editing experience will take. It might be something that is user-selectable, admin-selectable or something that can be enabled as a hybrid with the toolbar. Your thoughts on this approach are welcome!

Extending the editor​

All of this additional functionality is possible, not only with the strong foundations provided by both ProseMirror and Tiptap, but also through an extremely flexible and well thought out system enabling us to build our own functionality. By virtue of that, add-on developers are going to have a great experience bringing new features to our new editor.

For those of you less focused on development, this might not be quite as interesting. But let's just take a look at an example extension that adds some functionality to the editor:



While this is not (currently) shipping as a built-in feature (we haven't even created an associated BB code to render it in posts), we wanted to share it as an example of how easy and intuitive Tiptap is. Here's the sample Tiptap extension code we wrote that powers the functionality seen in the previous video:

JavaScript:
XF.EditorExtensions.registerExtension(({ Node, InputRule }) =>
{
    const inputRegex = /^!!!(highlight|important|success|warning|error)[\s\n]$/

    return Node.create({
        name: 'blockMessage',

        group: 'block',

        content: 'block+',

        defining: true,

        addAttributes()
        {
            return {
                style: {
                    default: 'highlight',
                    parseHTML: element => element.getAttribute('data-style'),
                    renderHTML: attributes =>
                    {
                        return {
                            'data-style': attributes.style,
                        }
                    },
                }
            }
        },

        parseHTML()
        {
            return [
                {
                    tag: 'div[data-type="block-message"]',
                },
            ]
        },

        renderHTML({ node })
        {
            return [
                'div',
                {
                    'data-style': node.attrs.style ?? 'highlight',
                    'class': 'blockMessage blockMessage--iconic blockMessage--' + (node.attrs.style ?? 'highlight'),
                },
                [
                    'div',
                    {},
                    0,
                ],
            ]
        },

        addInputRules()
        {
            return [
                new InputRule({
                    find: inputRegex,
                    handler: ({ state, range, match }) =>
                    {
                        const { from, to } = range
                        const tr = state.tr

                        const node = this.type.createAndFill({
                            style: match[1]?.trim(),
                        })
                        tr.replaceRangeWith(from, to, node)
                    },
                }),
            ]
        },
    })
})

A full breakdown of exactly how this works is beyond the scope of this post, but feel free to spend some time understanding it and the editor docs. If you have existing editor-based add-ons, the great news is that as long as you have some basic understanding of JavaScript, you should be able to produce your own extensions. We are not implementing any additional frameworks; everything is based on vanilla JavaScript.

One more thing...​

The astute amongst you might have already noticed that new lines work differently in the new editor compared to the current one, and there's a very good reason for that. We are pleased to announce that, finally, we are going to be using paragraph (<p>) tags properly starting with XenForo 2.4.

Hitting enter/return in the new editor now creates a new paragraph (with appropriate spacing). This may take a few moments to get used to, as hitting enter/return will now produce new line spacing equivalent to pressing enter/return twice currently. You can insert a "hard break" (producing a <br /> tag) with Shift + Enter. While this might require a brief adjustment, it is now consistent with how many applications and office suite tools work, so we don't expect it to take long.

The correct use of paragraphs doesn't stop with the editor, as we now correctly render content with paragraphs too!

HTML:
<h2 class="bbHeading">A brand new editor experience!</h2>
<div class="bbImageWrapper " title="Happy Season 9 GIF by The Office">
    <img src="https://media3.giphy.com/media/o75ajIFH0QnQC3nCeD/200.gif" class="bbImage">
</div>
<p>You might <b>recognise</b> the concept from applications such as <a href="https://notion.so" target="_blank">Notion</a>. You can add nodes from anywhere!</p>

<p>You can duplicate nodes. And move them around!</p>

<blockquote>
    <p>I've never had so much fun in an editor!</p>
</blockquote>

We haven't even changed the underlying BB code syntax. This is achieved with a brand new BB code parser which intelligently detects the correct boundaries for paragraphs and is able to distinguish between line breaks and paragraphs.

More coming soon!​

Thank you for joining us on this first look into the new editor we're implementing in XenForo 2.4. We're excited to get this in your hands in 2025. In the meantime we have some additional things for the editor that we're still wiring up:
  • Improvements to the developer experience for implementing custom editor functionality, BB code and custom buttons
  • Making use of the new editor elsewhere in XenForo
  • Further improvements to the UI and UX
  • Finalising the introduction of the experimental experience as an option
This will be the final Have you seen...? thread for 2024 and we will be back in the new year with even more to show you as we approach the final stages of this release.
 
It truly is a "what you see is what you get" experience, and to underline that even more, wait until you see what happens when you paste an embeddable link (best viewed full screen)
How does this affect GDPR compliance / cookie preferences?

Eg. what happens if a user that has rejected 3rd party cookies (=> embedded tweets in rendered posts would only show the consent placeholder) pastes a link to a tweet into the editor?
 
Looking forward to seeing this new editor.
Doesn't fuss me on when it drops!
All i know is we'll see it in our emails and see it on here once Chris and his crew of hard working developers have completed their task.
 
I just need to know the best place to propose these improvements and provide detailed feedback. Should I post them as suggestions on your community forum or report them as issues in a specific section?
I'd love to see XF staff will give you access to some private channel so your suggestions can be taken with more priority than usual suggestions
 
I'd love to see XF staff will give you access to some private channel so your suggestions can be taken with more priority than usual suggestions
 
Then again, I practically live in word processors a lot of the time outside work since I write for a hobby.
Same, although some of it is professionally. I feel I should not have to retrain myself each time I switch software. The essential keyboard shortcuts I have used since the 1990s in Word remain the same today, and many online text editors use the same ones. What I type or paste into an article is primarily what I expect as the output, without alteration. And I don't want to have to worry about using asterisks or any other characters modifying what I'm trying to input.

That said, I use Ctrl-Shift-V for a lot of my pasting...as that pastes in plain text, unformatted. In fact, it's such a habit now that I often have to undo (Ctrl-Z) and redo with Ctrl-V if I want to paste in something with formatting. 😁 Given the sources I paste from, data is often arranged in tables, or uses odd font styling or sizes, so plain text is the way I roll.

Most users are not familar with Markdown.
That is my biggest concern. The vast majority of forum users in any of those I manage wouldn't even know what Markdown is, and they're the type that would spend an hour or two not understanding why stray random characters are modifying their text, before reaching out to ask. For my purposes, Markdown is a non-starter for productivity reasons--otherwise, I would probably have to compose all the articles I publish in Word, then paste them into XF. But that's a "me" problem. I'm thinking more for others who just want to share their thoughts and are easily frustrated.

I have yet to really get accustomed to using that, though I have tried the text-to-speech in GBoard a couple times.
My biggest thing is...how do we enter punctuation? I'd love to use text-to-speech more (at least when I'm home alone) due to carpal tunnel issues and fatigue. I find it works OK on Gboard (and in the car through Android Auto) for replying to short messages but I can't see doing a 1,500 word article that way without a lot of cleanup after the fact.
 
The vast majority of forum users in any of those I manage wouldn't even know what Markdown is, and they're the type that would spend an hour or two not understanding why stray random characters are modifying their text
Slack and Discord both do this as you type, on desktop at least. (Can’t verify Slack on iOS, Discord does not on iOS but this is an iOS thing, not a Discord thing.)

I’d be a bit surprised if it was entirely alien, since using * for emphasis around words isn’t that uncommon, _ is not commonly typed in general, etc.

(Could be worse. Could be MediaWiki syntax which uses two single quotes either side of text for italics, and three single quotes either side for bold (so bold and italics is 5 single quotes either side of your text)

Another data point: one community I manage, not XF (and likely never to be unless I port a metric buttload of custom code), but it has people who write in Pages on iOS, on Word, and in one case in Notepad++, all doing bbcode inline in their posts and copy it in after. Writerly types are weird.
 
Another data point: one community I manage, not XF (and likely never to be unless I port a metric buttload of custom code), but it has people who write in Pages on iOS, on Word, and in one case in Notepad++, all doing bbcode inline in their posts and copy it in after. Writerly types are weird.
I actually still have a couple of longtime forum visitors or staffers who type out BBCode in their posts. For one of them, I can tell since he is somehow quoting posts using the bare minimum syntax, when he apparently doesn't see that he can highlight the selected text and click "Reply" on the little fly-out menu, or +Quote to cherry pick a handful of things to reply to.

On the flip side, the editor for one magazine I write for is not computer savvy. The magazine is using a blog component or plugin for Shopify (they are primarily a manufacturer who sells direct), so it has many of its own quirks. But luckily, it is friendly to copy/paste operations from Word so he doesn't have to fix too much after the fact.

Agreed about MediaWiki. I installed it once, about 15 years ago, to see if we could use it to compile an online collection. But it was too tedious for my staff to use, and that punctuation was certainly weird to adapt to as well. I was able to do something similar in WordPress, at least close enough that we made good use of it...but like with anything WordPress, the plugins I used for linking items were abandoned a decade ago. But at least there, the text editor was familiar enough for staff to use.
 
Slack and Discord both do this as you type, on desktop at least. (Can’t verify Slack on iOS, Discord does not on iOS but this is an iOS thing, not a Discord thing.)

I’d be a bit surprised if it was entirely alien, since using * for emphasis around words isn’t that uncommon, _ is not commonly typed in general, etc.

This is a good point. If anything, what with the introduction of this into mobile phone based message apps such as What'sApp, Discord and Slack, markdown is becoming far more popular than it ever was.

Mobile introduced the need to have inline bold and italics using markdown without UI buttons, so some people here may be quite surprised how many users are more familiar with it than they think.
 
Most users on our forums skew older and do not use social media or at the most, keep a FB account to stay in touch with family, and rarely post there.
 
The issue with “show who is typing” is the increased server use required. It’s not so bad if you have a server that can run Node or similar (or even something like Ratchet on PHP) where you can maintain an open connection to the server and push events at it. But PHP is fundamentally designed for “get a request, respond to it, close the connection” so if you’re doing that round trip, you’re making multiple database queries per time you phone home (for every user online)

You can stem the tide if you have multiple tabs open by using a message bus but still, for every user on the site, you need to phone home regularly (every few seconds) to get this data. And remember: you need to do this even if you’re just reading.

It’s not for nothing that Discourse (which does some of this) wants a 1GB VPS minimum and ideally a 4GB VPS for even a modest sized community - and even that you can cause various breakdowns relatively easily. (I speak from experience. Mind you, when you have a lead developer who proclaims loudly “there can’t be bots on this platform” only to witness multiple examples appearing the very next weekend… go figure.)

This is not insurmountable as a problem, but if you’re pushing server capacity now, you will almost certainly need to upgrade to support that.
 
But if it was there as an option with a caveat that it adds extra load it may be a feature that people find attractive.
Potentially. But on a busy forum that extra load is distinctly not trivial. Guarantee if it’s added there will be a slew of support issues where people are complaining how their site is slower now and/or their host is complaining.

There is no way to do this efficiently in the sort of hosting environment XF is aimed at. Sucks, but that’s a fact of life.
 
Back
Top Bottom