XF 2.4 XenForo 2.4 status and what's new under the hood?

Where are we?​

XenForo Community PSD Edit (1).webp
TL;DR: We're working hard to release XenForo 2.4 ASAP, but it's taking longer than expected due to scope changes and strategic decisions to wait for certain upstream developments that will benefit the long-term roadmap. Here's an analogy to explain why:

Software development is like planning a cross-country expedition with multiple destinations.

When you set out for version 2.4, you're not just driving to the next town over. You're charting a course through unknown territory with several strategic stops planned along the way - each representing a major milestone or feature release.

But the challenge is the landscape keeps changing along the journey.
  • New roads open up (better technologies emerge)
  • Bridges get washed out (dependencies break or become obsolete)
  • You discover scenic routes that would benefit all future travellers (opportunities for architectural improvements)
  • Weather conditions shift (market demands or user needs evolve)
  • Your vehicle needs unexpected maintenance (technical debt must be addressed)
You can't just focus on reaching the immediate next stop. You must consider how each decision affects the entire journey ahead. Taking a shortcut to reach 2.4 faster might leave you stranded when trying to reach 3.0, 4.0 or even 5.0.

This is why scope changes occur: experienced developers are constantly recalibrating the route based on new information, ensuring the expedition can successfully reach not just the next destination, but all the strategic waypoints that follow.

The delays aren't detours, rather they're course corrections that keep the long-term journey viable.

To be slightly less cryptic, these are some of the specific challenges we have faced along the way:

A new Tiptap version is coming​

When we announced that Tiptap is coming to XenForo 2.4 it was 95% complete, and we then took a bit of a pause to work on other projects, which we have talked about since and will be discussing in this thread. Since then, Tiptap have announced Tiptap V3 which is currently in beta. Given how core the editor is to the forum experience, it makes a lot of sense to ship XenForo 2.4 with Tiptap V3 rather than Tiptap V2 as originally planned. While the changes involved are not too extensive, we also don't want to ship 2.4 with a dependency that is still in beta and subject to change. While we are not planning to wait for Tiptap V3 to be stable, necessarily, we do at least want to give it a little bit more time so we have a higher degree of confidence that we're shipping a stable editing experience.

We started talking about a rewrite (again)​

While this is not currently the direction we've decided to go in, it's responsible for us to at least consider all routes available to us to help us reach our destination.

1749736697928.webp


After nearly 8 years since the release of XenForo 2.0, many of the technologies we use are showing their age, many of the decisions we made have started to slow us down more than we would like, and as a framework, XenForo becomes a less productive framework to work with. The solution to this problem can be to start from scratch, but we have ultimately decided that this is not something we need to do at this stage.

Instead, over the next few versions, including 2.4, we will be attempting to make iterative architectural changes to the framework so that we all have greater tools at our disposal to improve both the developer and user experience, particularly focusing on the implementation of developer tools and features that have become commonplace in other frameworks, such as Laravel.

Some of our best features are simply not finished​

There are one or two features that we see requested consistently from customers in our community forums and feedback channels, and we're excited to confirm they are coming in 2.4! However, it serves no one well if we release such highly-anticipated features before they are ready and before they have the usual level of quality, polish, and extensibility you would expect from a XenForo release. We'd rather take the extra time to get them right than rush them out and disappoint users with a subpar implementation that requires immediate patches or lacks the flexibility for customisation. We'll be sharing exciting details about what these features are and how they work in the coming weeks, so stay tuned!

We can't keep up!​

I just counted and there are about 15 features that have been merged or are pending to be merged into XF 2.4 that we haven't announced yet. Some of these are smaller and aren't worthy of a dedicated HYS of their own (so they'll probably be rolled into a "miscellaneous" HYS or two), and some of these are going to be mentioned below, but while we have been "cooking" (as the kids say these days) it has meant that things like code reviews, and writing HYS posts hasn't been easy to balance. There is also potentially more stuff coming from generous contributions from esteemed developers such as @Xon and @digitalpoint, assuming we have time to implement (otherwise they will wait for... a future version).


With all of that now being said, while 2.4 is taking longer than we wanted, we have been busy and we are very much nearing the end of development.

And, while disappointing (to all of us) it is important to maintain perspective. XenForo 2.2 was released in September 2020. XenForo 2.3 was released nearly four years later. XenForo 2.4 is not 3 more years away.

But, you clicked this to find out what's new, right? So let's go.
 
Last edited:
Click to expand...

Using PDO for database interactions rather than mysqli​

One of the things that we decided to do sooner, rather than later, was change our database library from mysqli to PDO. As we'll talk about later, PDO helps us write new database adapters more easily which becomes useful for supporting foreign databases for things like imports or, useful for third party developers who may wish to integrate with locally hosted foreign databases. PDO is also generally the de-facto standard for PHP at this point. Many hosts come with PDO enabled by default, whereas mysqli will often need to be enabled manually. We've seen this play out a couple of times when people change hosts or even change versions without ensuring the mysqli extension is enabled first.

To be able to install or upgrade to XenForo 2.4, you will need to ensure that PDO is available with the mysql driver in your hosting environment.

There are also some quality of life improvements included in PDO such as named parameter binding, for example:

PHP: 
$activeUsers = \XF::db()->fetchAll("
   SELECT *
    FROM xf_user AS user
    INNER JOIN xf_user_profile AS profile ON
        (user.user_id = profile.user_id)
    WHERE user.user_state = :user_state
        AND user.last_activity > :min_activity
    ORDER BY user.last_activity DESC
    LIMIT 10
", ['user_state' => 'valid', 'min_activity' => time()]);

Though, in retrospect, that may end up being less useful than you'd think... (more on that later).

Native support for SQLite​

SQLite is at this point the most widely deployed database in the world (across phones, embedded devices, apps etc.) with some even using it in large scale, high performance, production environments. SQLite is also great for developers due to its prevalence, zero-configuration setup, and portability (it's just a single file) with excellent performance on local disks. Despite its simplicity, SQLite is fully ACID compliant and remarkably robust. It is not just convenient, it's enterprise-grade reliable.

For advanced developers, particularly those with dev ops pipelines, SQLite lends itself extremely well to automated testing - you can spin up a complete, isolated database instance instantly for each test run. While we will still officially recommend MySQL for production, SQLite for testing and development is hugely attractive and we're excited to put it in your hands starting with XenForo 2.4.

But what about all of the existing queries in the code?

Brand new fluent query builder​

A new database means supporting a different database dialect. But no one wants to write database queries twice. In fact, it's a little bit annoying writing them once. The new query builder aims to make life easier. Let's rewrite the query I rewrote earlier, now using the new query builder:

PHP: 
$activeUsers = \XF::query('xf_user AS user')
    ->join('xf_user_profile AS profile', 'user.user_id', '=', 'profile.user_id')
    ->where('user.user_state', 'valid')
    ->where('user.last_activity', '>', time() - 86400)
    ->orderByDesc('user.last_activity')
    ->limit(10)
    ->fetchAll();

This query builder instance automatically compiles to the right SQL dialect for the database in use. We have rewritten all existing queries in the software (and official add-ons) to utilise the new query builder, including those generated by the Finder and the SchemaManager to support SQLite.

Of course you don't have to use it. You will be able to write different queries directly to support both MySQL (as now) and SQLite or you will be able to forget about SQLite altogether (though this may affect people's ability to use your add-on if they are using SQLite).

This also opens the door for supporting other databases in the future. While there are no immediate plans, support for MS SQL and PostgreSQL is now a possibility. Anyone who may need that right now will be able to easily write your own adapters and grammar classes for the query builder in order to add support if you need to.

A PhpStorm plugin for developers​

We acknowledge that occasionally query builders, while helpful, can sometimes slow development down because when writing raw database queries, IDEs such as PhpStorm are able to be configured to be "aware" of your database schema through its built-in database tools.

To address this issue, we're excited to announce we will be making available a free and open source PhpStorm plugin (please help us maintain it 🙏) for PhpStorm which provides completion for table and column names.

Here it is in action:



Debugging queries​

We have implemented a few methods to help developers check what query would be compiled by any particular query builder call for debugging purposes.

PHP: 
$pairs = \XF::query('xf_user')
    ->dump()
    ->select('user_id', 'username')
    ->limit(5)
    ->fetchPairs();

We support dump, dd, dumpSimple, ddSimple, dumpToFile, dumpToFileNamed methods anywhere in the query builder chain. These methods register an event that fires before the final query executes, meaning it also works with aggregate methods such as count and sum.

SQL: 
SELECT "user_id", "username" FROM "xf_user" LIMIT 5

Speaking of debugging queries - we've also given our debug output a refreshed look, now with better colour usage in both light and dark mode and syntax highlighting and formatting for database queries:

1749747940866.webp


Built-in PHP web server support​

Finding the right environment for development and testing can be challenging. There are a bunch of products available for different operating systems. Sometimes it can be difficult to find the right one for you, sometimes you might run into issues after upgrades, or you switch on your laptop one day and none of your databases are accessible. We've already talked about how SQLite can help with that - not having to manage your own MySQL or MariaDB instance is a huge benefit - but what if you didn't have to bother with Nginx or Apache, either?

Code: 
# php cmd.php xf:serve
Starting XenForo development server...
XenForo development server started: http://localhost:8080
Document root is: /Users/chrisdeeming/Herd/24x
Press Ctrl-C to quit.

13:26:20 GET 200 / (432.84ms)
13:26:21 GET 200 /css.php?css=public%3Anormalize.css%2Cpublic%3Afa.css%2Cpublic%3Avariations.less%2Cpublic%3Acore.less%2Cpublic%3Aapp.less&s=1&l=1&d=1747965566&k=975f8f0746a2cbf3a7f3ee7c96f538e9a2bf61e6 (488.67ms)
13:26:21 GET 200 /css.php?css=public%3Anode_list.less%2Cpublic%3Ashare_controls.less%2Cpublic%3Aextra.less&s=1&l=1&d=1747965566&k=b21bffa3ee848ff6a7384493bd4c8b387d4a511e (58.2ms)
13:26:21 GET 200 /js/xf/preamble.js?_v=e361daff
...

Utilising the PHP built-in web server, you can quickly spin up a development server without ever having to touch web server configs or full environments like Herd, WAMP, MAMP or Docker.

XenForo Development REPL​

A new xf-dev:tinker CLI command has been added, providing a REPL (read–eval–print loop) for XenForo development. It is built on the excellent Psysh library, and provides an easy playground for running code snippets, testing services, and inspecting the environment interactively.

The REPL is preloaded with common variables, like $app, $db, $em, and many others you might (not) expect. You can view the full list with the ls -l --vars command, or disable this behavior with the --without-vars flag.

You may use the --execute option to pass code to run directly, which can be useful for running quick one-off scripts, or pass file paths as arguments to include them when the REPL starts, which can be useful for shared set-up code. The REPL also includes a --with-commands flag to allow calling other XenForo CLI commands within the REPL.

The Psysh website and docs may be helpful references for getting started with it:
1749822698547.webp
 
Last edited:
Thanks Chris. Is there is a public beta of Tip Tap, and if so would it make sense to release the 2.4 beta with that or do you think it's best to wait for the Tiptap stable version so as to not confuse things?
 
Last edited:
Mr Lucky said:
Thanks Chris. Is there is a public beta of Tip Tap, and if so would it make sense to release the 2.4 beta with that or do you think it's best to wait for the Tiptap stable version so as to not confuse things?
Click to expand...
Sort of inferred in the HYS but essentially, yes, if we have to, the beta will contain a beta of Tiptap V3. Not against releasing 2.4 stable with the Tiptap beta either, once it is stable.
 
duderuud said:
On the one hand I understand waiting for new developments/changes in tech but on the other hand, you can wait indefinitely because there is always something around the corner.
Sounds to me like a reason to postpone 2.4 again for an even longer period.
Click to expand...
That's not quite what I said. One specific example was given which was involving Tiptap. Even if we waited for stable, that's not a long way. It's just sensible to not go all in on a beta version before it has had sufficient time to iron out some issues, and it's sensible to wait so we don't have to introduce potentially breaking changes. It's often about getting things done right rather than getting it done fast.
 
Chris D said:
Sometimes :)
Click to expand...
To be fair, this doesn’t help. I know, you can never satisfy everyone, and it’s natural to complain. But personally I’ve lost my trust in the development. There’s been too much talks, delays without actions.

Perhaps I’m wrong, and I say this all respectfully, but we need to see some actual progress and not yet another “HYS” coming soon feature that takes another months and months to come.

I do believe there’s so much more than what we see or hear, but some transparency and communication is long overdue.
 
JulianD said:
Is the development team making full use of all available copilots/cursors/windsurfs, and other AI code agents to enhance development speed and quality?
Click to expand...
Quality is certainly subjective. AI is a good tool for a number of things; it's good for prototyping ideas, it's quicker than Googling / reading StackOverflow, it's good at performing very specific tasks in an automated way. But often what you gain in those areas, if you try to push it beyond its limits, you end up losing time in nuance and edge cases and fixing things that AI doesn't always understand in the grander context.

But it is used by us where appropriate and practical.
 
CedricV said:
To be fair, this doesn’t help. I know, you can never satisfy everyone, and it’s natural to complain. But personally I’ve lost my trust in the development. There’s been too much talks, delays without actions.
Click to expand...
There's never "delays without actions". We quite literally never stop working. There's always action and progress.

CedricV said:
Perhaps I’m wrong, and I say this all respectfully, but we need to see some actual progress and not yet another “HYS” coming soon feature that takes another months and months to come.

I do believe there’s so much more than what we see or hear, but some transparency and communication is long overdue.
Click to expand...
This thread was progress, transparency and communication 👍
 
Fun morning read! I am still impressed by what the framework makes possible-especially from 2.2-2.3-and I feel very confident about its longevity if you still see so much to improve in the tooling that is "slowing you down" despite it being able to accomplish far more than basically any other CMS I've worked with.

Flashy features are great and XF is already full of them, but it's the depth and solid fundamentals that make me love this platform.
 
Back
Top Bottom