Fixed Emails can get stuck in the queue indefinitely

[PaulB]

Affected version

2.2.6 Patch 2

Currently, email queue processing works roughly like this:

1. Mark entry as being processed by setting send_date 15 minutes into the future.
   1. If marking failed, it's already being processed; skip this item.
2. Deserialize mail_data.
   1. If deserialization failed or is not an instance of Swift_Mime_SimpleMessage, skip this item without further changes.
3. Attempt to send the email.
   1. If send failed, Mailer is expected to call the queue's queueForRetry method.
   2. If send succeeds, delete the mail item from the queue.

The problem arises in step 2.1: if mail_data doesn't contain what the code expects, it silently fails, and the entry stays in the queue. No exception is logged and the mail stays where it is. Furthermore, fail_date and fail_count aren't touched. This can result in a situation whereby the environment or third-party code causes a hard-to-debug failure that prevents queued mail from being delivered and allows the mail queue to grow unchecked.

Note for people still on XF1: A similar bug exists there, but instead of the entry staying in the queue, it will be deleted.

 

[Chris D]

I can see that a failure to unserialize does indeed keep the message forever. Though I'm slightly more confused about why that would happen. Do you have a concrete example / reproduction case that you can share as that may well be the more significant issue.

 

[PaulB]

It was due to an issue on our end, and I would not expect the situation that we encountered to be happen to anyone else. I can send you details via DM, but it's probably not necessary. A more reasonable example might be an add-on developer who inadvertently corrupts a small fraction of emails, but nobody notices for quite a while because there aren't any short-term symptoms.

 

[Chris D]

I think in a lot of cases we probably wouldn't worry too much about this but the fix is trivial and I keep coming back to the point that a queue entry in this state is unlikely to be recoverable so it doesn't make sense to keep it around forever.

We're going to keep the queue entry in debug mode to aid in debugging, delete it otherwise, but in all cases we'll log an error.

 

[XF Bug Bot]

Thank you for reporting this issue, it has now been resolved. We are aiming to include any changes that have been made in a future XF release (2.2.7).

Change log:

Workaround an issue that could be caused by mail queue entries that fail to unserialize.

There may be a delay before changes are rolled out to the XenForo Community.