Lack of interest Update bounced email processing to correctly interpret certain Amazon SES/SNS messages

[SFFNetwork]

I recently migrated a forum from Mailgun to AWS SES for emails, and noticed that for bounce emails in which Amazon is reporting the status as 'inbox full' or 'suppressed email', XenForo doesn't interpret these as bounces but as 'Unknown' emails that are promptly discarded. I suspect that for any status that Amazon themselves will report back, the formatting prevents XenForo from interpreting them correctly.

I don't really consider this a bug because for more typical bounce notifications, XenForo interprets the notification correctly. That said, these are still bounces that Amazon penalizes customers for, as they're emails that are either temporarily (in the former case) or permanently (in the latter case) undeliverable. So it would be great if XenForo's bounced email processing was updated to interpret these notifications. Particularly as Amazon SES is far and away the most cost-effective solution for transactional email out there (that I know of at least).

Some (redacted) examples:

Inbox Full:

Date: Fri, 11 Dec 2020 10:37:08 +0000 From: --- To: --- Message-ID: <01000176516036ec-f774772e-9c6f-486e-9897-814b8f600ec5-000000@email.amazonses.com> Subject: AWS Notification Message MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit {"notificationType":"Bounce","bounce":{"feedbackId":"0100017651603592-75c47c02-f634-45c4-9ca8-b8568b3abf32-000000","bounceType":"Transient","bounceSubType":"MailboxFull","bouncedRecipients":[{"emailAddress":"example@gmail.com"}],"timestamp":"2020-12-11T10:37:08.000Z"},"mail":{"timestamp":"2020-12-11T10:32:25.125Z","source":"bounced+0aad7f3b+zen96737=example@---","sourceArn":"arn:aws:ses:us-east-1:520422078764:identity/bounced@---","sourceIp":"159.89.92.172","sendingAccountId":"520422078764","messageId":"01000176515be325-44a276c3-228b-4781-8cc9-732c50dcf0dd-000000","destination":["example@gmail.com"]}}

Suppression List:

Date: Tue, 15 Dec 2020 07:46:56 +0000 From: --- To: --- Message-ID: <01000176655dd257-d5c14603-5178-428b-9fb5-6087af031f6a-000000@email.amazonses.com> Subject: AWS Notification Message MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit {"notificationType":"Bounce","bounce":{"feedbackId":"01000176655dd17e-52fe24d8-3ef2-4c91-9e57-784ee759443d-000000","bounceType":"Permanent","bounceSubType":"OnAccountSuppressionList","bouncedRecipients":[{"emailAddress":"example@gmail.com","action":"failed","status":"5.1.1","diagnosticCode":"Amazon SES did not send the message to this address because it is on the suppression list for your account. For more information about removing addresses from the suppression list, see the Amazon SES Developer Guide at https://docs.aws.amazon.com/ses/latest/DeveloperGuide/sending-email-suppression-list.html"}],"timestamp":"2020-12-15T07:46:56.000Z","reportingMTA":"dns; amazonses.com"},"mail":{"timestamp":"2020-12-15T07:46:55.729Z","source":"bounced+fcb62bb8+example=gmail.com@---","sourceArn":"arn:aws:ses:us-east-1:520422078764:identity/bounced@---","sourceIp":"---","sendingAccountId":"520422078764","messageId":"01000176655dd071-8e4860c0-b33e-42c6-b713-0b3591ce8431-000000","destination":["example@gmail.com"]}}

 

[notifications]

TL;DR: If you’re using Amazon SES via SNS, XenForo may treat some bounce notifications as “Unknown”. Two practical paths that work today:
1) Enable SES Email Feedback Forwarding (no SNS) → XenForo parses the standard DSN just fine.
2) Keep SNS → normalize SES JSON “bounce” types (e.g., MailboxFull → Soft, OnAccountSuppressionList → Hard) before handing them to XenForo.

Context
-------
We’ve seen SES/SNS payloads where:

• "bounceType": "Permanent", "bounceSubType": "OnAccountSuppressionList" should be treated as a hard bounce (suppress permanently).
• "bounceType": "Transient", "bounceSubType": "MailboxFull" should be treated as a soft bounce (retry with backoff).

When these land as “Unknown”, addresses might not be suppressed or retried correctly.

Path A — No SNS (simplest)
--------------------------

• In SES console, for your verified identity, enable Email Feedback Forwarding.
• Point the forwarding to the mailbox XenForo already polls for bounces (IMAP/POP).
• Result: XenForo consumes the DSN and updates user email states without extra plumbing.

Path B — Keep SNS (JSON)
------------------------

• Keep your SNS Topic + HTTPS subscription.
• In your receiver endpoint, normalize SES payloads and map to XenForo semantics:

{
  "notificationType": "Bounce",
  "bounce": {
    "bounceType": "Transient",
    "bounceSubType": "MailboxFull",
    "bouncedRecipients": [{ "emailAddress": "user@example.com" }]
  }
}

Recommended mapping (examples):

• Transient + MailboxFull → Soft bounce (do NOT permanently suppress; implement retries).
• Permanent + OnAccountSuppressionList → Hard bounce (suppress immediately).
• Permanent + Suppressed/DNS/Invalid → Hard bounce.
• Transient + Throttling/Timeout → Soft bounce (temporary).

If you maintain a small middleware, something like:

<?php
// Pseudo-code: SES SNS -> XenForo-friendly action
$payload = json_decode($raw, true);

if (($payload['notificationType'] ?? '') === 'Bounce') {
    $b = $payload['bounce'] ?? [];
    $type = $b['bounceType'] ?? '';
    $sub  = $b['bounceSubType'] ?? '';
    $recips = array_column($b['bouncedRecipients'] ?? [], 'emailAddress');

    $action = 'unknown';
    if ($type === 'Permanent') {
        $action = 'hard';
    }
    if ($type === 'Permanent' && in_array($sub, ['OnAccountSuppressionList','Suppressed','General'])) {
        $action = 'hard';
    }
    if ($type === 'Transient' && in_array($sub, ['MailboxFull','Throttling','Timeout','TemporaryFailure'])) {
        $action = 'soft';
    }

    foreach ($recips as $email) {
        // call into XenForo / your job queue:
        // hard => mark invalid / suppress; soft => increment soft count & retry later
        queueBounceAction($email, $action, $type, $sub, $payload);
    }
}

Why this helps
--------------

• Preserves XenForo’s expected behavior (hard vs soft decisions) even when SES uses nuanced subtypes.
• Prevents accidental permanent suppression for temporary conditions (MailboxFull).
• Ensures truly permanent failures (Suppression list, invalid addresses) are removed promptly.

Real-world note
---------------
We’ve been running this approach in production at vevioz.com with good results: soft bounces decay correctly, and permanent ones are suppressed immediately. No promotional intent—just sharing an implementation detail that might help others hitting the same edge cases.

If core wants to support SES/SNS JSON directly, the key is to interpret bounceType + bounceSubType pairs as above. Until then, either Feedback Forwarding (no SNS) or a tiny JSON normalizer keeps things reliable.