SparkPost Mail Transport for XF 2.3 5.1.0

Adds a SparkPost mail transport option to have all mails sent via your SparkPost account.

This is the XF v2.3 compatible version of SparkPost Mail Transport for XF 2.2 and has been completely rewritten to support Symfony Mail that is used in XF v2.3

This addon has full bounced email and unsubscribed email handling support using the SparkPost Events API - no mailbox required.

Compatibility

Version 5.x of this addon requires XenForo 2.3.x and PHP 8.3 or later - do not try to install it on an earlier version of either.

If your server is on PHP 8.1 or 8.2, the upgrade will be refused and your existing version will keep working - nothing is changed. Version 4.x supports PHP 8.1 and above.


You may upgrade between addon versions once you have upgraded your forum version.

Upgrading

Important: before upgrading your site to XenForo 2.3, you should disable the SparkPost Mail Transport v2.x addon on your XF 2.2 site. Failing to do so will likely break your site after upgrading to XF 2.3.

If your site does break after upgrading to XenForo 2.3, disable addons by setting the following value in your config.php file - then upgrade the SparkPost addon to the XF 2.3 version (SparkPost Mail Transport v5.x) - then remove the setting from your config.php file:

$config['enableListeners'] = false;

If you are still running the What's New Digest adapter for SparkPost addon (Hampel/WndSparkPost), this addon will refuse to install until you uninstall it. That adapter has been incompatible since v3.1.0, when What's New Digest handling moved into this addon directly.

Setup

If you haven't already done so, you will need to configure a sending domain for your forum in your SparkPost account. It is highly recommended that you also set up a default bounce domain.

You will then need to create an API key with the following permissions:

• Events Search: Read-only
• Transmissions: Read/Write

Once you have your API key, change the Email transport method in the Email options page to "SparkPost" and paste your API key into the supplied box.










Configuration

As per the screenshot above - you may configure some transport options when configuring SparkPost.

• Click Tracking turns on click tracking on all links in emails sent
• Open Tracking adds a tracking image to all emails sent to track opens
• Test Mode delivers every message to '.sink.sparkpostmail.com' instead of the real recipient, so you can test communication with SparkPost without actually sending any emails to your users. Since v5.0.0 this is done by rewriting the SMTP envelope, so the message itself still reads as addressed to the real recipient - previously the sink suffix was visible to whoever received one.

Note that it is highly recommended that you set up a custom tracking domain before enabling click or open tracking.

There is an additional configuration page under the SparkPost Mail option heading:

• Message Events Batch Size allows you to configure how many events the system will attempt to retrieve in each batch. In general you shouldn't need to adjust this from the default.

Testing

You can test the functionality of the addon using the built in XenForo email test system - go to the admin UI > Tools > Test outbound email



Note the email transport class should be Hampel\SparkPost\Transport\SparkPostTransport - if it isn't, then you haven't installed or activated the addon correctly. (In v4.x and earlier this was Hampel\Symfony\Mailer\SparkPost\Transport\SparkPostApiTransport.)

There is also a second test facility which allows you to specify whether to send transaction or non-transactional emails. Go to the admin UI > Tools > Test SparkPost




Turn on the checkbox to send transactional email - turn it off to send non-transactional.

In all cases, you should examine the headers of the emails you receive to verify that it was indeed sent via SparkPost and to validate the presence/absence of the List-Unsubscribe header (the header should be present for non-transactional emails and absent for transactional emails).

Checking your configuration from the command line

Since v5.1.0 there are two commands that answer "is this set up" and "does it work" without needing the admin UI:

php cmd.php sparkpost:config reports what your forum is set up to do and where each value came from - the selected transport, whether an API key is set and which host it points at, test mode, tracking, the message event batch size and when events were last fetched. It never prints your API key, so the output is safe to paste into a support thread.

php cmd.php sparkpost:validate then proves your forum can actually do it: the wiring, the message event table, the cron entries and the fetch watermark, followed by one authenticated read from SparkPost to confirm your API key is accepted. It sends no mail.

• it exits 0 when everything is fine, and 1 if anything failed - so sparkpost:validate && ... can gate a deployment
• --unattended skips the call to SparkPost, for a scheduled check with nobody watching
• --strict exits 2 when there are warnings but no failures

Transactional vs Non-transactional Emails

Transactional emails are automated emails sent by a system to communicate with a customer or site user about information relevant to their account or usage of your site. They are typically specific to an individual user and are generally important to the user for the ongoing use of your site.

Examples of transactional emails are:

• account creation and activation emails
• password resets and two-factor authentication
• security and account alerts
• notifications resulting from subscriptions or alert options the user has chosen
• purchase receipts and shipping notifications
• legal and important site notices

Non-transactional emails (also referred to as Commercial Emails) are generally sent to a group of users in bulk (although they may also be personalised), containing promotional or marketing material. Usually only users who have opted in to receive such material from you will receive them.

Examples of non-transactional emails are:

• newsletters
• sales offers
• new product updates
• email designed to increase user engagement and onboarding, etc

The SparkPost website has more information about Transactional vs Commercial emails.

By default, SparkPost marks all emails sent through their API as "non-transactional". This addon changes that behaviour and sets all forum emails to transactional except for:

• Emails sent via the admin UI - Communication > Email users
• Welcome emails
• What's New Digest emails, if you have the What's New Digest addon installed

There are two significant differences between transactional and non-transactional emails sent via SparkPost:

1. non-transactional emails have the List-Unsubscribe header added (overrides the header added by XenForo) for one-click unsubscribe requests in some email clients. Transactional emails do not contain this header.
2. there are separate suppression lists for transactional and non-transactional emails. If someone unsubscribes from non-transactional emails, you can still send them important transactional emails like forum subscriptions and password resets. However, if someone unsubscribes or blocks you from transactional emails, you should disable their account, since you will no longer be able to email them about important forum information.

See below for more information on suppression lists.

Bounce and Unsubscribe Handling

The automated bounce and unsubscribe handling in the addon will check for all of the standard hard and soft bounce calls that might be received as a result of sending emails. These bounced messages are intercepted by SparkPost and we can then execute a query to determine what action we need to take as a result.

You do NOT need to configure a "Bounced email address" (unless you don't have a default bounce domain set up) or "Unsubscribe email address" in your email options. Nor do you need to enable the "Automated email handler" or "Enable automated email handler" options - SparkPost does not forward bounced emails or unsubscribe notifications via email - we query their API to determine bounce / unsubscribe information.

Unsubscribe handling is a little more sophisticated than the standard XenForo system - since we differentiate between transactional and non-transactional emails and handle them differently.

Note that if we receive a hard bounce for transactional emails - the users account will be disabled to prevent further emails being sent.

Auto-replies (such as out-of-office responses) and opt-in confirmations are not treated as delivery failures - since v5.0.0 these are recognised as messages that reached the recipient, and no action is taken against their account.

Suppression lists

When SparkPost receives a hard bounce from a sent email or a List-Unsubscribe request, it will place that email address onto a suppression list which means that you can no longer send email to that user. This is in place to protect your sending reputation - since continuing to send email to an invalid or unsubscribed address is a spam indicator which could see you blocked from some mail providers.

The thing to be aware of is that if a users forum account has been disabled due to bounced emails, they will most likely have been added to the SparkPost suppression list as well. If you re-enable their account without changing email addresses, then they may not receive any emails from your site because sending to them is blocked by SparkPost.

If you are sure the email is now valid, you can remove them from the SparkPost suppression list - in the SparkPost admin console - https://app.sparkpost.com/lists/suppressions.

Just remember that there are two separate lists - one for transactional emails and one for non-transactional emails.