No email bounce log action from Amazon SES spam complaints

[Mouth]

When a user marks email received from your site as 'spam' in their mail application, the majority of mail applications will then send a spam/abuse notification back to the sending server complaints process.

For Amazon SES, they then send/forward an email to you of the spam/abuse complaint initiated by the recipient to your Bounced Email Address. Refer https://sesblog.amazon.com/blog/tag/abuse+complaint

When xF processes this spam/abuse complaint to your bounces inbox, no action is taken.



As per above blog, I believe it should be processed and mark the user state as email invalid (bounced). (Or even better, an additional user state of "email invalid (spam report)")

Here's a sample spam/abuse report email ("<redacted>" added by me) that comes to xf bounces inbox ...

Delivered-To: bounces+7893a4dc+<redacted>=yahoo.com.au@<redacted>.com.au
Received: by 10.202.171.74 with SMTP id u71csp470804oie; Fri, 13 May 2016
14:40:02 -0700 (PDT)
X-Received: by 10.66.152.201 with SMTP id va9mr26178294pab.73.1463175602646;
Fri, 13 May 2016 14:40:02 -0700 (PDT)
Return-Path: <no-reply@amazonses.com>
Received: from a27-9.smtp-out.us-west-2.amazonses.com
(a27-9.smtp-out.us-west-2.amazonses.com. [54.240.27.9]) by mx.google.com with
ESMTPS id a3si26505755pfb.217.2016.05.13.14.40.02 for
<bounces+7893a4dc+<redacted>=yahoo.com.au@<redacted>.com.au> (version=TLS1
cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 13 May 2016 14:40:02 -0700
(PDT)
Received-SPF: pass (google.com: domain of no-reply@amazonses.com designates
54.240.27.9 as permitted sender) client-ip=54.240.27.9;
Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify)
header.i=@arf.mail.yahoo.com; spf=pass (google.com: domain of
no-reply@amazonses.com designates 54.240.27.9 as permitted sender)
smtp.mailfrom=no-reply@amazonses.com
X-Original-To: complaints@email-abuse.amazonaws.com
Delivered-To: complaints@email-abuse.amazonaws.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arf.mail.yahoo.com;
s=arf; t=1463175600; bh=JA0DfLoWSnvhMk580J67urAR0K7gCZqQ9TKYs4ruNEA=;
h=Date:From:To:Subject:From:Subject;
b=YK7XkRpp5vjWWJ2wTJMNqL6FOmPrYqdyclIrBL68go84/DCB6BEURpYNnDQ+sTIH47JJtcFQYX9VO1nfqPYj6W+QjcksfxSUjnrbIBn+dZo4haV9vyuiyUHtDYGYOoS5cuAU5KFFe5xnKOFK+wp6ctW4i2EglEO8rC/u542Z1fM=
X-Yahoo-Newman-Id: cfl-test
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: multipart/report; boundary="_----------=_14631756006709";
report-type="feedback-report"
X-Mailer: MIME::Lite 3.028 (F2.82; A2.11; B3.13; Q3.13)
Date: Fri, 13 May 2016 21:40:01 +0000
From: complaints@us-west-2.email-abuse.amazonses.com
To: bounces+7893a4dc+<redacted>=yahoo.com.au@<redacted>.com.au
X-Yahoo-Newman-Property: cfl
Message-ID: <01010154ac0fdc08-3f04390a-1953-11e6-bf3a-518ba1613008-000000@us-west-2.amazonses.com>
X-SES-Outgoing: 2016.05.13-54.240.27.9

This is a multi-part message in MIME format.

--_----------=_14631756006709
MIME-Version: 1.0
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="US-ASCII"
X-Mailer: MIME::Lite 3.028 (F2.82; A2.11; B3.13; Q3.13)
Date: Fri, 13 May 2016 21:40:00 +0000

This is an email abuse report for an email message received from 54.240.27.34  on Fri, 13 May 2016 16:39:32 UTC

--_----------=_14631756006709
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
Content-Type: message/feedback-report

Feedback-Type: abuse
User-Agent: Yahoo!-Mail-Feedback/1.0
Version: 0.1
Original-Mail-From: <01010154aafcb9d8-90945e40-33ed-4290-ba4a-4bf13e2bef7d-000000@us-west-2.amazonses.com>
Original-Rcpt-To: <redacted>@yahoo.com.au
Received-Date: Fri, 13 May 2016 16:39:32 UTC
Source-IP: 54.240.27.34

 

[Mike]

This isn't a bounce indication, so this is the correct behavior. This isn't something we would expect to cover in a bounced email handler. As such, I think this is more correctly considered a suggestion.

What Amazon is doing here isn't 100% the "standard", though it's a reasonable approach. In a direct feedback loop, this feedback would be sent to a specific address (see various examples here: http://wiki.asrg.sp.am/wiki/Feedback_loop_links_for_some_email_providers). Presumably Amazon is parsing out the feedback report and the original envelope sender and sending the feedback there. With VERP, we are able to identify the original email at least which would allow an action, but without that, there isn't an action we would take (as we can't verify the report as XenForo-triggered).

 

[Mouth]

This isn't a bounce indication

Yes, technically.
But I believe the result and action should be the same handling. If someone is initiating a complaint about receiving your site email, then treatment and resulting action should be the same.

 

[Mike]

My secondary point is also that in general, feedback look emails go to a standardized address (based on sender or IP of the sender), rather than back the original envelope sender. This appears to be something Amazon is doing specifically. I'd be interested to know if any other SMTP providers do this (and whether they include the original email, as it appears Amazon doesn't).

There's also ambiguity in the action that you might want to take. If someone is making a spam complaint for a watched thread notification, stopping their email notifications for watched threads may be what should technically be done. (I take that if they're doing this, they probably don't care about your site and thus it likely doesn't matter what action is taken as long as the emails stop.)

 

[Mouth]

Here's another example, bounce email from Amazon SES but not being recognised/processed by XF ...

Delivered-To: <redacted>
Received: by 10.202.117.68 with SMTP id q65<redacted>41oic; Thu, 7 Jul 2016
00:24:52 -0700 (PDT)
X-Received: by 10.98.16.156 with SMTP id 28mr277<redacted>876291896; Thu,
07 Jul 2016 00:24:51 -0700 (PDT)
Return-Path: <01010155c43eb2fc-<redacted>-258779af5cb5-000000@us-west-2.amazonses.com>
Received: from a27-84.smtp-out.us-west-2.amazonses.com
(a27-84.smtp-out.us-west-2.amazonses.com. [54.240.27.84]) by mx.google.com
with ESMTPS id n17si2816949pfi.210.2016.07.07.00.24.51 for
<redacted> (version=TLS1 cipher=ECDHE-RSA-AES128-SHA
bits=128/128); Thu, 07 Jul 2016 00:24:51 -0700 (PDT)
Received-SPF: pass (google.com: domain of
01010155c43eb2fc-<redacted>-000000@us-west-2.amazonses.com
designates 54.240.27.84 as permitted sender) client-ip=54.240.27.84;
Authentication-Results: mx.google.com; dkim=pass header.i=@amazonses.com;
spf=pass (google.com: domain of
01010155c43eb2fc-<redacted>-000000@us-west-2.amazonses.com
designates 54.240.27.84 as permitted sender)
smtp.mailfrom=01010155c43eb2fc-<redacted>-000000@us-west-2.amazonses.com
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
s=gdwg2y<redacted>5hhxx; d=amazonses.com; t=1467876291;
h=Date:From:To:Message-ID:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID;
bh=/cHQVG<redacted>VvrODJko=;
b=ivTo3U6ik<redacted>cdDwBDDvn8tqF
Tn78xpicCiDP6<redacted>B4GAkCLh7OPDr
ONra+Q<redacted>m8+StEVCzHk0=
Date: Thu, 7 Jul 2016 07:24:51 +0000
From: Netrider Bounces <no-reply@sns.amazonaws.com>
To: <redacted>
Message-ID: <01010155c43eb2fc-<redacted>-000000@us-west-2.amazonses.com>
Subject: AWS Notification Message
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
x-amz-sns-message-id: ee5f79<redacted>acd0e8583c
x-amz-sns-subscription-arn: arn:aws:sns:us-west-2:403791450681:netrider-bounces:ac8d<redacted>5adb44e
X-SES-Outgoing: 2016.07.07-54.240.27.84
Feedback-ID: 1.us-west-2.c55J8LO2<redacted>80AUVC/G8=:AmazonSES

{"notificationType":"Bounce","bounce":{"bounceType":"Permanent","bounceSubType":"General","bouncedRecipients":[{"emailAddress":"cmb84_<redacted>eva@hotmail.com","action":"failed","status":"5.3.0","diagnosticCode":"smtp; 550 Requested action not taken: mailbox unavailable"}],"timestamp":"2016-07-07T07:24:51.196Z","feedbackId":"01010155c43eb2<redacted>7f05f-000000","reportingMTA":"dsn; a27-116.smtp-out.us-west-2.amazonses.com"},"mail":{"timestamp":"2016-07-07T07:24:49.000Z","source":"bounces+e3cb9fd8+cmb84_<redacted>eva=hotmail.com@netrider.com.au","sourceArn":"arn:aws:ses:us-west-2:40379<redacted>m.au","sendingAccountId":"403791450681","messageId":"01010155c43eab02-<redacted>d-000000","destination":["cmb84_<redacted>eva@hotmail.com"]}}

--
If you wish to stop receiving notifications from this topic, please click or visit the link below to unsubscribe:
https://sns.us-west-2.amazonaws.com/unsubscribe.html?SubscriptionArn=arn:aws:sns:us-west-2:403<redacted>28-9b56-4a72d5adb44e&Endpoint=<redacted>

Please do not reply directly to this email. If you have any questions or comments regarding this email, please contact us at https://aws.amazon.com/support

 

[AppleAndMango]

@Mouth Did you figure this out?

I'm having the same issue...

 

[Mouth]

@Mouth Did you figure this out?
I'm having the same issue...

I opened an official support ticket with XF, and they said it's not something they wish to support. Thus, only partial support for Amazon SES by XenForo.

I now use https://xenforo.com/community/resources/bd-mails.2313/ which handles them appropriately.

 

[Jeremy P]

Bounce notifications from SES work for me using the stock handlers, with nothing special configured on the SES side either. If you configure SNS email notifications for bounces and complaints in SES, XF will not be able to process them. If you just allow for regular delivery of these reports (not SNS), XF seems to ensure the reports get sent to the right place and can process them fine.

 

[AppleAndMango]

I opened an official support ticket with XF, and they said it's not something they wish to support. Thus, only partial support for Amazon SES by XenForo.

I now use https://xenforo.com/community/resources/bd-mails.2313/ which handles them appropriately.

I'm using this currently too...

Bounce notifications from SES work for me using the stock handlers, with nothing special configured on the SES side either. If you configure SNS email notifications for bounces and complaints in SES, XF will not be able to process them. If you just allow for regular delivery of these reports (not SNS), XF seems to ensure the reports get sent to the right place and can process them fine.

I'll give this a shot, thanks.

 

[Mouth]

I'll give this a shot, thanks.

Be aware you'll only get hard/soft bounce handling. Spam complaints are still not handled.

 

[ProCom]

We recently converted a big board over to XF and have been sending about 6,000 emails a day and getting about 20 bounces per day. Unfortunately we haven't been properly processing all the bounces. Not a huge problem at this low bounce rate, but we're about to send out our first newsletter since our conversion to XF and I'm concerned that our bounce rate will be super high and no way to properly process all the bounce notifications from SES.

Be aware you'll only get hard/soft bounce handling. Spam complaints are still not handled.

The number of these that we get are relatively low, so we could, in theory, process these manually. It's the thousands of bounces we'll get when we send the newsletter that I'm worried about.

 

[Mouth]

It's the thousands of bounces we'll get when we send the newsletter that I'm worried about.

AWS will likely stop and ban your SES usage, if that's the case.
I'd suggest you get the email list cleaned first. https://xenforo.com/community/threads/amazon-ses-and-xenforo.128853/

 

[BlingMuncher]

Bounce notifications from SES work for me using the stock handlers, with nothing special configured on the SES side either. If you configure SNS email notifications for bounces and complaints in SES, XF will not be able to process them. If you just allow for regular delivery of these reports (not SNS), XF seems to ensure the reports get sent to the right place and can process them fine.

Sorry to bump this old thread, but I'm having the issue where XF is picking up the bounced mail but not detecting the address or taking any action.
My setup is SES > SNS > bounced inbox > XF Pickup.

If you configure SNS email notifications for bounces and complaints in SES, XF will not be able to process them. If you just allow for regular delivery of these reports (not SNS), XF seems to ensure the reports get sent to the right place and can process them fine.

What do you mean by this? How can xenforo process bounces without a mailbox to read?

Once again, sorry for the necropost, but help is needed :|

Edit:
Jesus chirst I've just spent 6+ hours trying to figure this out, and I realise that it has been working this whole time but the bounced emails are going into gmail spam folder, so xenforo doesn't see them...
I was doing all this complicated stuff with SNS and trying different transport modes for the bounce box and crap.
It's actually very straightforward....

 

[trapped_soul]

Am having some issues myself and was wondering if anyone can help.

Been using SES for some time (2 years I think) but can't get the bounce working correctly.
I am not getting the actual notifications from SES.. Does anyone have a step by step guide for this?
here's what I have set up;
SES three emails, all domains verified and DKIM.
XF sending;
Default Sending address myemails@mydomain.com

Bounced Email Address bouncedemails@mydomain.com
Enable automated bounced email handler
IMAP;
imap@mydomain.com
Port 993
User Name and Password: bouncedemails@mydomain.com / mypassword

Then in SES I have
aws:ses:ABCD-1:23456789:identity bouncedemails@mydomain.com
Email Feedback Forwarding: enabled

But nothing comes in at all. Have I missed a step? Does SES automatically forward bounces from the XF sent emails, to the email address listed from within SES or do I need to set up something else with the handler?
I'm lost with this now as I think I've done the same as @BlingMuncher - been at it for so long have totally confused myself I think.

 

[trapped_soul]

I opened an official support ticket with XF, and they said it's not something they wish to support. Thus, only partial support for Amazon SES by XenForo.

I now use https://xenforo.com/community/resources/bd-mails.2313/ which handles them appropriately.

Hi @Mouth can I ask, how are you managing to use that for bounces? Do you still need to enter the AWS keys etc?

 

[trapped_soul]

Okay I followed this and had some serious issues. I also clicked delete on the posts I made by accident but they're not too relevant.
If you are struggling to do the same in this thread;
https://xenforo.com/community/threads/amazon-ses-and-xenforo.128853/
Then I strongly suggest not to use your own domain host for the emails, but follow what some others have done and use Gmail then add filters so it's not spam.
That works, 100%

 

[BlingMuncher]

Okay I followed this and had some serious issues. I also clicked delete on the posts I made by accident but they're not too relevant.
If you are struggling to do the same in this thread;
https://xenforo.com/community/threads/amazon-ses-and-xenforo.128853/
Then I strongly suggest not to use your own domain host for the emails, but follow what some others have done and use Gmail then add filters so it's not spam.
That works, 100%

Is it working for you now?
I only use custom domain for sending, and then use a gmail account for bounces, works like a charm.

 

[trapped_soul]

Is it working for you now?
I only use custom domain for sending, and then use a gmail account for bounces, works like a charm.

Thank you, yes it is now. The only way to do it for me, was by using Gmail for bounces. Our domain email wouldn't play.
So we have domain for sending and Gmail for bounces, all through SES without using SNS (for bounces anyway).
Taken hours to sort and when switching to Gmail after seeing so many other replies using Gmail for bounces, it worked within seconds.

 

[trapped_soul]

Sigh ... am now getting imap login details incorrect even though less secure apps is enabled etc etc
Hmm I wonder if it's OAuth2 issues as I get a lot of problems with people registering with Twitter/FB too

 

[DeltaHF]

I was just hit by this issue as well. A very active user put my forum's email notifications on "ignore" (that's what he said when I asked him about it, and apparently that's equivalent to a spam complaint with Hotmail.co.uk). This sent my SES spam complaint rate above 0.5% in a matter of days and got my Amazon AWS account on probation.

I am dealing with the issue in two ways:

• Set a weekly reminder to check my XenForo Email Bounce Log for any suspicions "Unknown", no-action entries.
• In my dedicated bounced email account (using Gmail), I created a filter to forward all messages from complaints@email-abuse.amazonses.com to my personal email address and to never send those messages to spam. This will notify me immediately if this starts to happen again. As mentioned in the discussions above, it's very important to ensure bounced email notifications or any emails from SES are never marked as spam.

I would strongly advise anyone using SES with XenForo's Bounced Email features to do the same.