No email bounce log action from Amazon SES spam complaints

[Mouth]

When a user marks email received from your site as 'spam' in their mail application, the majority of mail applications will then send a spam/abuse notification back to the sending server complaints process.

For Amazon SES, they then send/forward an email to you of the spam/abuse complaint initiated by the recipient to your Bounced Email Address. Refer https://sesblog.amazon.com/blog/tag/abuse+complaint

When xF processes this spam/abuse complaint to your bounces inbox, no action is taken.



As per above blog, I believe it should be processed and mark the user state as email invalid (bounced). (Or even better, an additional user state of "email invalid (spam report)")

Here's a sample spam/abuse report email ("<redacted>" added by me) that comes to xf bounces inbox ...

Delivered-To: bounces+7893a4dc+<redacted>=yahoo.com.au@<redacted>.com.au
Received: by 10.202.171.74 with SMTP id u71csp470804oie; Fri, 13 May 2016
14:40:02 -0700 (PDT)
X-Received: by 10.66.152.201 with SMTP id va9mr26178294pab.73.1463175602646;
Fri, 13 May 2016 14:40:02 -0700 (PDT)
Return-Path: <no-reply@amazonses.com>
Received: from a27-9.smtp-out.us-west-2.amazonses.com
(a27-9.smtp-out.us-west-2.amazonses.com. [54.240.27.9]) by mx.google.com with
ESMTPS id a3si26505755pfb.217.2016.05.13.14.40.02 for
<bounces+7893a4dc+<redacted>=yahoo.com.au@<redacted>.com.au> (version=TLS1
cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 13 May 2016 14:40:02 -0700
(PDT)
Received-SPF: pass (google.com: domain of no-reply@amazonses.com designates
54.240.27.9 as permitted sender) client-ip=54.240.27.9;
Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify)
header.i=@arf.mail.yahoo.com; spf=pass (google.com: domain of
no-reply@amazonses.com designates 54.240.27.9 as permitted sender)
smtp.mailfrom=no-reply@amazonses.com
X-Original-To: complaints@email-abuse.amazonaws.com
Delivered-To: complaints@email-abuse.amazonaws.com
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arf.mail.yahoo.com;
s=arf; t=1463175600; bh=JA0DfLoWSnvhMk580J67urAR0K7gCZqQ9TKYs4ruNEA=;
h=Date:From:To:Subject:From:Subject;
b=YK7XkRpp5vjWWJ2wTJMNqL6FOmPrYqdyclIrBL68go84/DCB6BEURpYNnDQ+sTIH47JJtcFQYX9VO1nfqPYj6W+QjcksfxSUjnrbIBn+dZo4haV9vyuiyUHtDYGYOoS5cuAU5KFFe5xnKOFK+wp6ctW4i2EglEO8rC/u542Z1fM=
X-Yahoo-Newman-Id: cfl-test
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: multipart/report; boundary="_----------=_14631756006709";
report-type="feedback-report"
X-Mailer: MIME::Lite 3.028 (F2.82; A2.11; B3.13; Q3.13)
Date: Fri, 13 May 2016 21:40:01 +0000
From: complaints@us-west-2.email-abuse.amazonses.com
To: bounces+7893a4dc+<redacted>=yahoo.com.au@<redacted>.com.au
X-Yahoo-Newman-Property: cfl
Message-ID: <01010154ac0fdc08-3f04390a-1953-11e6-bf3a-518ba1613008-000000@us-west-2.amazonses.com>
X-SES-Outgoing: 2016.05.13-54.240.27.9

This is a multi-part message in MIME format.

--_----------=_14631756006709
MIME-Version: 1.0
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="US-ASCII"
X-Mailer: MIME::Lite 3.028 (F2.82; A2.11; B3.13; Q3.13)
Date: Fri, 13 May 2016 21:40:00 +0000

This is an email abuse report for an email message received from 54.240.27.34  on Fri, 13 May 2016 16:39:32 UTC

--_----------=_14631756006709
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
Content-Type: message/feedback-report

Feedback-Type: abuse
User-Agent: Yahoo!-Mail-Feedback/1.0
Version: 0.1
Original-Mail-From: <01010154aafcb9d8-90945e40-33ed-4290-ba4a-4bf13e2bef7d-000000@us-west-2.amazonses.com>
Original-Rcpt-To: <redacted>@yahoo.com.au
Received-Date: Fri, 13 May 2016 16:39:32 UTC
Source-IP: 54.240.27.34

 

[eva2000]

I was just hit by this issue as well. A very active user put my forum's email notifications on "ignore" (that's what he said when I asked him about it, and apparently that's equivalent to a spam complaint with Hotmail.co.uk). This sent my SES spam complaint rate above 0.5% in a matter of days and got my Amazon AWS account on probation.

I am dealing with the issue in two ways:

• Set a weekly reminder to check my XenForo Email Bounce Log for any suspicions "Unknown", no-action entries.
• In my dedicated bounced email account (using Gmail), I created a filter to forward all messages from complaints@email-abuse.amazonses.com to my personal email address and to never send those messages to spam. This will notify me immediately if this starts to happen again. As mentioned in the discussions above, it's very important to ensure bounced email notifications or any emails from SES are never marked as spam.

I would strongly advise anyone using SES with XenForo's Bounced Email features to do the same.

Amazon Cloudwatch alerts are the key https://community.centminmod.com/th...tp-transactional-email-info.13842/#post-62892

 

[Alpha1]

I would love to see this implemented so that bounce handling works for SES / SNS.

 

[Alpha1]

Yes, technically.
But I believe the result and action should be the same handling. If someone is initiating a complaint about receiving your site email, then treatment and resulting action should be the same.

I somewhat disagree with this. Yes, there should be no more email sent to the user. But more action needs to be taken than with a normal bounce. We also display a notice to users with bounced status and the text of this notice is not correct because the situation is different. Its not the case that the provider of the user is simply refusing the email.

The member user is actively filing a complaint against the site. When the user returns to the site he should see a message that confront him with his actions and instructs him how to correct his actions before ever using the site again. If the user is simply able to change email address then this may cause more complaints. I don't want to see that happen. And if that happens I want to know it so I can ban the member.

 

[Joe Link]

Any updated info on how people are handling these complaints?

 

[DeltaHF]

I am still using the same system that I mentioned previously. Fingers crossed, haven't had any issues since then.

 

[BoostN]

If you can set and endpoint in the SNS to your XF installation, build an addon to ingest that webhook data to transform and mark the profile as bounced it would solve this issue correct?

 

[Joe Link]

I somewhat disagree with this. Yes, there should be no more email sent to the user. But more action needs to be taken than with a normal bounce. We also display a notice to users with bounced status and the text of this notice is not correct because the situation is different. Its not the case that the provider of the user is simply refusing the email.

The member user is actively filing a complaint against the site. When the user returns to the site he should see a message that confront him with his actions and instructs him how to correct his actions before ever using the site again. If the user is simply able to change email address then this may cause more complaints. I don't want to see that happen. And if that happens I want to know it so I can ban the member.

I absolutely agree with this, there should be a separate user state for complaints.

 

[Solari]

I was just hit by this issue as well. A very active user put my forum's email notifications on "ignore" (that's what he said when I asked him about it, and apparently that's equivalent to a spam complaint with Hotmail.co.uk). This sent my SES spam complaint rate above 0.5% in a matter of days and got my Amazon AWS account on probation.

I am dealing with the issue in two ways:

• Set a weekly reminder to check my XenForo Email Bounce Log for any suspicions "Unknown", no-action entries.
• In my dedicated bounced email account (using Gmail), I created a filter to forward all messages from complaints@email-abuse.amazonses.com to my personal email address and to never send those messages to spam. This will notify me immediately if this starts to happen again. As mentioned in the discussions above, it's very important to ensure bounced email notifications or any emails from SES are never marked as spam.

I would strongly advise anyone using SES with XenForo's Bounced Email features to do the same.

Great advice. I also included SMS notification via Amazon SNS in the topic subscription for complaint emails just to make absolutely sure I get them and act on 'em right away to disable the complainer's email on XF.

I'm kinda surprised there is no mechanism in XF to handle complaints automatically. To at least turn off emails to their accounts. In this day and age it's a pretty critical feature. Am curious if it's on the roadmap somewhere?

Ray

 

[Alpha1]

Am curious if it's on the roadmap somewhere?

I dont think so. There is this suggestion for complaints statistics and it has come up in various discussions and suggestions, but there is no suggestion for a complaints handler.

Statistics for Email Sent, Bounces & Complaints

Its important to see if your bounce rate is getting high. Because a high bounce rate means your email can get blocked by email providers or even your own email provider if you have one. Providers like Amazon will ban you if your bounce rate gets too high. Email services like yahoo will greylist...

xenforo.com

 

[ProCom]

We're still dealing with this issue:

1. Somebody signs up for the forum, subscribes, starts posting.
2. They get an email from the site that someone replied
3. Out of confusion or laziness, they hit the SPAM report button in gmail, etc.
4. Amazon received the complaint & emails our account
5. We have to check and make sure we manually take action on our end

It would be GREAT if SES email complaints were handled automatically!

In the interim, I'm not sure how to proceed. Mark the user's account as "email invalid"?

What happens when the confused member doesn't know why they can't login? If they try to reset their password or try to contact us using the contact form, and we respond, our emails will be blocked since they were (at one time) marked as spam.

How to resolve this?

How to convey to the member (outside of email) that they need to un-mark our emails as spam?

 

[DeltaHF]

We're still dealing with this issue:

1. Somebody signs up for the forum, subscribes, starts posting.
2. They get an email from the site that someone replied
3. Out of confusion or laziness, they hit the SPAM report button in gmail, etc.
4. Amazon received the complaint & emails our account
5. We have to check and make sure we manually take action on our end

It would be GREAT if SES email complaints were handled automatically!

In the interim, I'm not sure how to proceed. Mark the user's account as "email invalid"?

What happens when the confused member doesn't know why they can't login? If they try to reset their password or try to contact us using the contact form, and we respond, our emails will be blocked since they were (at one time) marked as spam.

How to resolve this?

How to convey to the member (outside of email) that they need to un-mark our emails as spam?

If it's any consolation, you're not alone. I've been jumping through those same hoops for years. It's a real problem for which there doesn't seem to be any real fix.

You should still be able to communicate with your users through email, though. SES is for "transactional" emails only, so you should still be using another email provider (like Google Workspace) to send your own messages. I'll admit this doesn't always work, though, and I have had my replies blocked as well. There's really nothing more to do at that point.

 

[sub_ubi]

This is still working for me, though Amazon may have changed something as newer users are having problems with setup.

Beta - Amazon SES Bounces Support

Amazon SES Bounces Support This addon creates an endpoint in your forum that will be used by AWS SNS to send notifications about bounces and complaints. Since it doesn't creates the notification topics automatically it is your job to create them...

xenforo.com

 

[ProCom]

SES is for "transactional" emails only, so you should still be using another email provider (like Google Workspace) to send your own messages.

Good point. We are using Google Workspace to send out the emails, so in theory, that could resolve the issue around correspondence (assuming they don't spam-report those emails too, LOL).

 

[MaximilianKohler]

Wow. 8 years later this has still not been fixed. It's disturbing that such a basic and required feature gets ignored for so long.

It's an inherent feature for forums to send out email notifications. They need a built-in method to automatically process spam complaints. There was even an addon linked previously in this thread for XF v1.x. Why haven't the XF developers integrated that into Xenforo for version 2.x?

Currently, the only 2.x option/solution is a 3rd party addon only for Amazon SES.

 

[Kirby]

There is no standardized way (yet) for email senders to receive complaint reports.

So while it would certainly be possibly to implement smth. proprietary for SES, I can totally understand why XenForo doesn't really want to do that:
Implementing proprietary technology in most cases isn't a great idea, many customers most likely won't have significant issues with complaints and last but not leat SES isn't used by everyone (probably not even a majority of customers) anyway.

But there is ongoing effort to establish a standard and it seems like this might be getting some traction, at least it was a topic for the last Certified Senders Alliance summit.

So if RFC 9477 starts to be supported by major mailbox providers (like Google, Microsoft and Yahoo) I am pretty sure XenForo might be more willing to implement support for complaint management, just like they implemented support for One-Click Unsubscribe although this suggestion was initially kinda rejected.

 

[MaximilianKohler]

There is no standardized way (yet) for email senders to receive complaint reports.

Not by using a dedicated bounce email perhaps. But all the major software I've come across whose main function is to send email, all have a built-in way to handle bounces and complaints. And many of them are open source, such as listmonk, mautic, etc. They use the method that the SES addon linked two posts above uses. I guess it's called "webhook".

Using a dedicated email to handle it seems strange.

So while it would certainly be possibly to implement smth. proprietary for SES

As I said, someone previously linked to an XF 1.x addon in this thread that did it for more than just SES.

last but not leat SES isn't used by everyone (probably not even a majority of customers) anyway

That was one of my points, so I think you didn't understand my reply well.

 

[DeltaHF]

I see we can update user_state via the API.

I suppose we could write our own webhook endpoint which edits the target user via the API.