XF 2.3 Miscellaneous changes for XenForo 2.3

Due to time constraints and family matters, this is a slightly different HYS to what was planned, but it is still a bumper feast of new features coming in XenForo 2.3. This week we will be mostly focusing on a bunch of smaller new changes and improvements we're no less excited to show you in the near future.

This is a somewhat lengthy post so we will say goodbye here and we will see you for more 2.3 goodies next week.

Sign in with Apple​

To join our existing suite of connected account providers, in XenForo 2.3 we are adding support for Sign in with Apple. The set up for this one will be a little more onerous, requiring an Apple developer account and the creation of a certificate file that will need to be uploaded through our UI:

hys_6_apple_1.png


Before release, we will be documenting the full setup process through the XenForo manual. There is also an additional step required for users who sign up with Apple's "Hide my e-mail" feature.

Once enabled, users will be able to sign in, or sign up with the Apple button in the relevant places.

hys_6_apple_continue.png


Search users for connected accounts​

If you've ever wondered which of your users have which connected accounts connected, you can now pull that list directly from the "Search users" page in your admin control panel.

hys_6_user_search.png


Simply select the specific connected account(s) you want to search for, and the list of users who have those accounts connected will be displayed.

IndexNow support​

In XenForo 2.3 you can now enable support for IndexNow. This is a recent initiative by Bing, Yandex and others, which allows you to directly notify them whenever content is created, updated, or deleted on your forum. Whenever one of these events happens, a job is enqueued to submit the URL to IndexNow. This avoids the need for generating huge sitemap files which may or may not be read by search engines.

Due to limited support by search engines, including Google, the sitemap generation remains in place and remains unchanged, but IndexNow is available for those search engines which support it either now or in the future.

Generically queued and retryable jobs​

Starting with XenForo 2.3, the existing job system has been enhanced with additional tracking which allows it to be used as a generic queueing system. In XenForo 2.2, mails are added to a special queue table before being processed. This functionality remains largely unchanged, but instead of queueing mails to be sent in their own table, they are now queued directly into the xf_job table.

It's not just mails which are queued in this way. The sending of push notifications and webhooks are also now queued into the job system, so there is little to no noticeable delay when submitting content which might generate one or more of these to be sent.

In addition to queueing these items in this way, developers can also opt in to having their jobs be retryable in the event of failure. This maintains the current behaviour of the existing mail queue and also allows webhooks to be retried in the event of failure.

The cool off between retrying can be configured directly in the Job class or use sensible defaults so that the delay between retries is lengthened based on the number of previous attempts.

After a certain number of failed attempts, the jobs can be marked as failed and they are stored in a new xf_failed_job table.

Bundled support for remote object storage​

Thanks to some changes in the AWS SDK for PHP we are now able to ship, alongside XenForo, the parts of the library which are responsible for communicating with Amazon S3 and compatible object storage services.

The setup instructions provided by the below resource are still required:

But starting with XenForo 2.3 you will no longer need to install an "add-on" in order to configure it. Calling it an "add-on" is a little bit of a misnomer as all it really does is include the (previously) humongous AWS SDK. Now we can include just the Amazon S3 SDK parts, there are fewer steps required to offloading your storage to a remote service, like Amazon S3, Cloudflare R2 and many others.

Full InnoDB support with improved MySQL search​

In days gone by, there were good reasons for our continued use of MyISAM and MEMORY tables in MySQL. Though as the years have progressed, the need for these storage engines in databases have long been negated by better hardware performance and improvements in InnoDB.

Starting with XenForo 2.3 we are automatically converting most of the remaining tables to use InnoDB and for new installs we are making all tables use InnoDB by default.

The only table that requires manually converting is the xf_search_index table. For existing installations, due to differences in the full-text search implementation, you will need to run a CLI command named xf:convert-search-innodb, which will empty, convert and optionally rebuild the search index for you.

Full text search using InnoDB should be an improvement right out of the gate as it offers a lower default minimum word length and a much smaller set of stop-words, along with an improved scoring algorithm. We now even support relevancy search order out of the box, something that was previously a XenForo Enhanced Search exclusive feature, though this will be subject to real world testing before we can comment on whether it is useful.

Due to these changes we are going to be requiring a minimum of MySQL 5.7 or MariaDB 10.2 starting with XenForo 2.3.

Native 'sticky' and date/time inputs​

Due to ancient browser quirks which should no longer be relevant and improved HTML standards, we are now handling sticky elements and date inputs natively without the use of third party libraries.

In terms of 'sticky' support, this affects mostly the admin control panel navigation and the sticky header on the public side in some legacy cases. For the most part, you shouldn't see any difference and the experience, particularly for the admin control panel navigation, should be a lot less janky in some of the cases.

Date inputs within XenForo have now been replaced with native date inputs, rather than using a third party library. Each browser/OS will render the date picker slightly differently, as seen below (Chrome, Firefox, macOS Safari and iOS pictured):

hys_6_date_chrome.png
hys_6_date_firefox.png
hys_6_date_safari.png
hys_6_date_apple.png

But all of them are at least much more functional than they used to be and will be familiar to people who have used the date inputs before.

In addition to supporting date inputs with the existing <xf:dateinput> tag, we also support date and time inputs with the <xf:datetime> tag and time inputs using the <xf:timeinput> tag, which correspond to the HTML standard <input type="date">, <input type="datetime-local"> and <input type="time"> tags respectively.

Auto refresh "board inactive" page​

For those times when you need to turn your forum off, you will have used the "board is active" toggle to do so. This page will now automatically refresh every 60 seconds so when the forum is back up, people will be back using your site sooner without needing to hit refresh.

Anchor links for headings​

Every time a heading is used within content, it now has an automatic anchor link generated which can be accessed on hover via the link icon that appears next to the heading:

hys_6_heading.png


This will allow you to link directly to headings within your content.

Email notifications for moderators​

Sometimes it can be tricky keeping up to date with the various goings on with a busy forum, or if you're a small team, sometimes it can be difficult to respond to various moderator tasks.

To make that easier in XenForo 2.3, moderators can now opt-in to email alerts for new reported content or new content awaiting approval,

hys_6_moderator_emails.png


With these preferences enabled (on a per-moderator basis under account preferences), whenever any of those actions require attention, an email will be sent to those moderators.

User ID matches expression​

New for 2.3 is an addition to the user criteria selector that allows CSS-style :nth-child selectors against user IDs. While this might initially sound like gobbledygook, there is a powerful application for this tool.

1698099926187.png


Let's imagine that you have a user group promotion that grants access to a 'Testing group' user group, and applies to all users with a 2n value for 'User ID matches expression. This promotion will apply to any user whose user ID is even. You can then enable specific functionality for users in the 'Testing group', and use various tools to measure their engagement with those tools, compared to those not in that group who do not have access to the changed functionality. In short, you have a quick and easy way to do A/B testing on your entire user base.

Of course you don't have to limit your testing to one group. You could set up 3n, 3n+1 and 3n+2 criteria on three separate promotions to divide your user base into three groups, or set up segregation however you like.
 
Last edited:
Can you elaborate on the performance implications of the newly enhanced job system, especially in forums with high traffic? How does the queuing mechanism handle surges in activities like mail sending and push notifications without affecting user experience?
 
Can you elaborate on the performance implications of the newly enhanced job system, especially in forums with high traffic? How does the queuing mechanism handle surges in activities like mail sending and push notifications without affecting user experience?
I think there is a follow up on that coming as well.
 
Full text search using InnoDB should be an improvement right out of the gate as it offers a lower default minimum word length and a much smaller set of stop-words, along with an improved scoring algorithm. We now even support relevancy search order out of the box, something that was previously a XenForo Enhanced Search exclusive feature, though this will be subject to real world testing before we can comment on whether it is useful.
How does this compare with ElasticSearch in terms of performance of use of resources?
 
Ultimately I would expect MySQL full text search, even when utilising InnoDB as its engine, will still not be a great option when compared to Elasticsearch, particularly with large / busy forums. Though it will very likely be an improvement over the existing MyISAM based full-text search.

There's an overhead to running Elasticsearch because it's an additional service running, with its own storage requirements, so there is that to factor in, but I would expect it will have an overall net advantage over MySQL.
 
An email newsletter queued to 1,000,000 users was already problematic with it blocking emails that are more immediately important (like a password reset email), but now it also potentially blocks other non-email jobs.

Funny you should say that but that is basically the solution @Naz has been planning (it was noted down but didn't make it to the backlog)

I've got a question along these lines. I'm certainly never sending 1,000,000 emails at a time but I do send to up to 1,000 at a time using the Communication -> Email users option in the admin CP. My question is, will we somehow be able to limit the number of those emails that are sent per minute with this new job system?

I only tried it once and I got a ton of errors from my server about sending too many at once, and I ended up getting my IP address temporarily blocked from Yahoo and Gmail for the same reason. So, I haven't done it since.

That said, I have a specific user group that it really would be super nice to be able to send an email to a few times a year and there's really no way to do it now in XF without getting blocked by everyone.
 
I've got a question along these lines. I'm certainly never sending 1,000,000 emails at a time but I do send to up to 1,000 at a time using the Communication -> Email users option in the admin CP. My question is, will we somehow be able to limit the number of those emails that are sent per minute with this new job system?

I only tried it once and I got a ton of errors from my server about sending too many at once, and I ended up getting my IP address temporarily blocked from Yahoo and Gmail for the same reason. So, I haven't done it since.

That said, I have a specific user group that it really would be super nice to be able to send an email to a few times a year and there's really no way to do it now in XF without getting blocked by everyone.
Better to be controlled at the mail server level, not the application. It depends on your mail server, but as an example, in Postfix there are some config options like:

Code:
smtp_destination_concurrency_limit = 1
smtp_destination_rate_delay = 10s
smtp_extra_recipient_limit = 10

That's going to give you better control over rate limiting outbound email. Email servers from different domains don't talk to each other, so if you were to send 10,000 emails at once, all to different domains, there's no issue sending them all at once. It's really when you start sending to the same domain where it becomes problematic. And those config options will tell your MTA to slow down to the same domain (10 seconds between each being sent to gmail.com or yahoo.com for example). But the email server will do what it's supposed to do on the backend... accept the emails for delivery and then deliver them as instructed.

Just blindly adding 10s between emails when it may not even be necessary (depending on the receivers domain) will cause everything to be slow even when it doesn't have to be.

TL;DR: Tell the mail transport agent how fast emails should be delivered, that's literally what it's designed for.
 
Better to be controlled at the mail server level, not the application. It depends on your mail server, but as an example, in Postfix there are some config options like:

Code:
smtp_destination_concurrency_limit = 1
smtp_destination_rate_delay = 10s
smtp_extra_recipient_limit = 10

That's going to give you better control over rate limiting outbound email. Email servers from different domains don't talk to each other, so if you were to send 10,000 emails at once, all to different domains, there's no issue sending them all at once. It's really when you start sending to the same domain where it becomes problematic. And those config options will tell your MTA to slow down to the same domain (10 seconds between each being sent to gmail.com or yahoo.com for example). But the email server will do what it's supposed to do on the backend... accept the emails for delivery and then deliver them as instructed.

Just blindly adding 10s between emails when it may not even be necessary (depending on the receivers domain) will cause everything to be slow even when it doesn't have to be.

TL;DR: Tell the mail transport agent how fast emails should be delivered, that's literally what it's designed for.

I had a feeling this is what the reply would be. Unfortunately, you're speaking a language that I don't understand. 😂

Sounds like I need to start reading up on how to properly set up my server for sending emails because, yes, what you're saying sounds like what I should be doing.

Thanks for the reply, even though you're not telling me what I want to hear. 😳 😂
 
I had a feeling this is what the reply would be. Unfortunately, you're speaking a language that I don't understand. 😂

Sounds like I need to start reading up on how to properly set up my server for sending emails because, yes, what you're saying sounds like what I should be doing.

Thanks for the reply, even though you're not telling me what I want to hear. 😳 😂
Sorry, but ya... it's the better way to do it. Delivering email and how you want to do that is literally the reason there is a MTA (mail transfer agent) and not applications bypassing the MTA and delivering directly to destination mail servers via SMTP. While you could just add a hard pause between emails at the application-level, that's so inefficient it's crazy and you are also artificially trying to keep a potentially super long running process at the application level that is really just sitting there and doing nothing (doing it that way is effectively replacing your really good MTA with a crappy/archaic one... just need to learn how to use your MTA). Doing it at the MTA level allows you to deliver emails fast, except for a situation where you are delivering a zillion to one specific mail server... you can slow those down to keep that mail server happy.

Which I guess I pretty much already said all that... hah
 
I had a feeling this is what the reply would be. Unfortunately, you're speaking a language that I don't understand. 😂

@digitalpoint tried to tell you this (with more words):
While it is possible (to some degree) to handle queuing in the application, I think this isn't the best place to do this - queuing mail for delivery should be the job of a MTA.

Really, if you are sending more than a handful emails - use a provider that does this properly or at least use an intermediate MTA (that delivers via smarthost).
Reinventing the wheel at application level doesn't make sense (and only screams for trouble).
 
Top Bottom