XF 2.2 Configure Amazon SES in our Xenforo forum

[Sadiq6210]

Hi

I created new amazon SES account, verified the domain, created SMTP, and moved the account out of the sandbox. I think that our SES is ready.

Then in Email options, I changed the method to SMTP with SES information:







Now when I try to send test email through Contact page, I got this error in admincp and the message is not delivered to "Contact email address"

Swift_TransportException: Email to ***@***.com from ***@***.com failed: Expected response code 250 but got code "554", with message "554 Message rejected: Email address is not verified. The following identities failed the check in region US-EAST-1: *** <****@live.com> " src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php:459
Generated by: *** 25 Apr 2022 at 12:29 PM
Stack trace
#0 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(344): Swift_Transport_AbstractSmtpTransport->assertResponseCode('554 Message rej...', Array)
#1 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/EsmtpTransport.php(305): Swift_Transport_AbstractSmtpTransport->executeCommand('
.
', Array, Array, false, NULL)
#2 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(406): Swift_Transport_EsmtpTransport->executeCommand('
.
', Array)
#3 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(502): Swift_Transport_AbstractSmtpTransport->streamMessage(Object(Swift_Message))
#4 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(518): Swift_Transport_AbstractSmtpTransport->doMailTransaction(Object(Swift_Message), 'support+74cb150...', Array, Array)
#5 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(206): Swift_Transport_AbstractSmtpTransport->sendTo(Object(Swift_Message), 'support+74cb150...', Array, Array)
#6 src/XF/Mail/Mailer.php(312): Swift_Transport_AbstractSmtpTransport->send(Object(Swift_Message), Array)
#7 src/XF/Mail/Mail.php(458): XF\Mail\Mailer->send(Object(Swift_Message), Object(XF\Mail\SmtpTransport), NULL, true)
#8 src/XF/Service/Contact.php(197): XF\Mail\Mail->send()
#9 src/XF/Pub/Controller/Misc.php(86): XF\Service\Contact->send()
#10 src/addons/OzzModz/ContactusLog/XF/Pub/Controller/Misc.php(87): XF\Pub\Controller\Misc->actionContact()
#11 src/XF/Mvc/Dispatcher.php(352): OzzModz\ContactusLog\XF\Pub\Controller\Misc->actionContact(Object(XF\Mvc\ParameterBag))
#12 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Misc', 'Contact', Object(XF\Mvc\RouteMatch), Object(OzzModz\ContactusLog\XF\Pub\Controller\Misc), NULL)
#13 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(OzzModz\ContactusLog\XF\Pub\Controller\Misc), NULL)
#14 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#15 src/XF/App.php(2352): XF\Mvc\Dispatcher->run()
#16 src/XF.php(524): XF\App->run()
#17 index.php(20): XF::runApp('XF\\Pub\\App')
#18 {main}
Request state
array(4) {
  ["url"] => string(19) "/forum/misc/contact"
  ["referrer"] => string(41) "https://www.***.com/forum/misc/contact"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(7) {
    ["_xfToken"] => string(8) "********"
    ["subject"] => string(6) "test2b"
    ["message"] => string(14) "test2b message"
    ["_xfRedirect"] => string(29) "https://www.***.com/forum/"
    ["_xfRequestUri"] => string(19) "/forum/misc/contact"
    ["_xfWithData"] => string(1) "1"
    ["_xfResponseType"] => string(4) "json"
  }
}

Did I miss anything in configuration?

 

[TPerry]

I gave xen a dedicated mailbox to handle the bounce with the plugin. Using both worked well.

Yah, it was easier for me to take the add-on out of the mix. The add-on is what is supposed to handle the bounce, and if you are already using a dedicated mailbox, then no need for the add-on since SES was returning those bounced messages.

 

[MentaL]

Yah, it was easier for me to take the add-on out of the mix. The add-on is what is supposed to handle the bounce, and if you are already using a dedicated mailbox, then no need for the add-on since SES was returning those bounced messages.

it filters out the requests and blocks them and manages your reputation effectively.

 

[TPerry]

it filters out the requests and blocks them and manages your reputation effectively.

Pretty sure the SES system already handles the bounces and they are in an exclusion list so they don't get sent again without use of that add-on. That add-on was basically so that you didn't need dedicated bounce/unsubscribe email accounts, it was handled using the SES reporting and the API. It was the API not reporting back to XF that I had issues with. So I found no need to use it since the dedicated email accounts handled that fine, and SES returns those bounced emails to them, having the XF account disabled for email bounces and SES listing it as a bounced exclusion to not send to again.

 

[ActorMike]

That Amazon stuff is overpriced, and they charge like 30 bucks/month for a dedicated IP address which is 1/2 the cost of a server that comes with 5 IP addresses. See- https://bit.ly/DediOutlet If you are running on a shared address, you can still have problems!

We send out 20k messages at 100+ per second on our own dedicated Windows server, and they always get delivered to the inbox because I have test accounts with @aol @Yahoo @hotmail @gmail and we watch the feedback loop and ban clowns that report forum messages as spam.

No reason to pay Amazon to send your messages!

 

[TPerry]

Most admins are not going to take the time and effort to learn how to set up their own mail server and keep their reputation clean. It's easier using a provider like Amazon SES. And no, the cost is not that high if all you are using it for is the sending of transactional emails.
That's like saying "There's no reason to pay XenForo to host your site on their cloud offering". Yes, there is a reason. Not everyone has the same abilities. And if you don't, then you have to pay someone to administer that server for you, adding into that cost.
I have NO issues delivering to any email service on the shared hosting environment (Microsoft, Yahoo, ComCast and many others either). And so far, at my usage level I have had to pay zero to Amazon for the use of the service. Even if I have to end up paying $20 a month, I'm still ahead.

 

[MaximilianKohler]

SES is $10 for 100k emails. I don't think most people need their own $30 IP.

 

[ActorMike]

So I replied with this:

Additional information for production request:

• XXX.com sends email messages daily to users of the XXX.com discussion forum.
• The recipient mailing list is maintained in our Xenforo software. The software sends messages related to password reset, registration verification, and messages pertaining to specific forum events which users can choose to receive or not receive.
• Email bounces are processed by the Xenforo software. Currently bounced emails are considered a “bounce” for 2 unsuccessful delivery attempts over a 5 day period. After the 2nd unsuccessful delivery attempt the user is moved to a “Email invalid” state and no further emails are sent until the user changes his/her email address. The software also informs the user of such action via a message to that user placed on the forum itself, not via email.
• Unsubscribes are also handled by the Xenforo software. All outgoing messages have an unsubscribe option. For unsubscribe requests, the software adjusts the user’s profile so that further email messages are not sent to that user’s email address. Complaints would be in the same category as an unsubscribe since any user can unsubscribe to unwanted emails. A complaint is basically an unsubscribe request.
• Generally, we have had good results sending email messages with our current arrangement except for Yahoo.com and AOL.com. Apparently, some of the servers used by our website shared server hosting arrangement are blacklisted by Yahoo/AOL which is a primary reason we are moving to Amazon SES.
• Attached is a sample outgoing message that is bounced for reason "spam" by a small percentage of email servers.

I copied & pasted that on our application and it worked the first try. LOL Took 8 days to get approved! 50,000 emails per 24-hour period Maximum send rate 14 emails per second.

I'm going to test it out and have it as a backup method in the event we have email server issues.

 

[TPerry]

Now when I try to send test email through Contact page, I got this error in admincp and the message is not delivered to "Contact email address"

If you are talking about the "Contact Us" link, disable your Sender info in from header on contact emails option in the ACP-Setup-Options-Email section.
I had issues with the Contact Us for guest users (signed in users worked fine) until I disabled that setting.

 

[avclub]

Hello,

Do you know what is the maximum send rate through smtp from xenforo?
I have asked amazon and raised the limit to 45 emails per sec, would it be enough to not get throttled?

 

[TPerry]

50 emails per second (my allowance) are quite a bit... I doubt you are going to get throttled at that rate. Of course, if you find that you are, you can always ask for an increase, explaining why.

 

[ActorMike]

50 emails per second (my allowance) are quite a bit... I doubt you are going to get throttled at that rate. Of course, if you find that you are, you can always ask for an increase, explaining why.

How did you get them to change it to 50 emails per second? I requested an increase and at first they refused, then I told them, we'll just use our own email server then, which we've used for over 20 years. Then they said we will do 25 emails per second.

We sent our email to members each week to 13,000 people is that not enough to justify an increase?

Also, noticed the Amazon IP is blacklisted on HOSTKARMA-BLACK AND SORBS-RECENT so we'll probably just switch back to our own email server as it's not on ANY blacklists at all.

 

[TPerry]

How did you get them to change it to 50 emails per second? I requested an increase and at first they refused, then I told them, we'll just use our own email server then, which we've used for over 20 years. Then they said we will do 25 emails per second.

I submitted a simple request to increase my limits, and I repeated the fact that it was transactional emails from a forum and the process for bounce/unsubscribe handling that was in place. A few hours later they increased it.
Of course, this was about 1.5 years ago when they weren't as strict as they have apparently become.

I just checked, and now it's back down to 25... so they made a change on my account sometime recently. I can still request again if I needed it, but I can't justify that high of a rate for what I send currently.

 

[GW2]

We sent our email to members each week to 13,000 people is that not enough to justify an increase?

I don't understand the need for such high rates of sending email messages, i.e. 50 per second.

13,000 email messages sent at 25 messages per second requires 520 seconds, or about 9 minutes. There may be factors such as internet bandwidth that could increase that 9 minutes...but significantly? Is there an urgency that requires that all 13,000 emails be sent at such a high speed?

 

[ActorMike]

I don't understand the need for such high rates of sending email messages, i.e. 50 per second.

13,000 email messages requires 520 seconds, or about 9 minutes.

The real question is, why should it be throttled starting at 14 messages per second and you have to beg to get 25? When I send the messages out on our server with our own mail server ,the messages blast out in no time.

I noticed the Amazon IP is blacklisted on HOSTKARMA-BLACK AND SORBS-RECENT so we'll probably just switch back to our own email server as it's not on ANY blacklists at all.

If you look at some 7 year old reviews of AMZ SES they once sent out at 3000 messages per second!

PS We don't do slow around here!

 

[TPerry]

The real question is, why should it be throttled starting at 14 messages per second and you have to beg to get 25? When I send the messages out on our server with our own mail server ,the messages blast out in no time.

Really.. that's not unexpected in the ultimate realm of things. VERY few sites are going to need 50+ per second sends in the ultimate realm of things.. and most of those that do will have grown with SES and they will recognize those needs.

I noticed the Amazon IP is blacklisted on HOSTKARMA-BLACK AND SORBS-RECENT so we'll probably just switch back to our own email server as it's not on ANY blacklists at all.

That will happen with ANY MTA provider that offers the same IP to multiple clients at a low cost. SES is not exempt from spammers using them.. although they do a very valid job of killing them when found. This would apply to ANY provider of mail services, not only SES.


Once more.. if you have the staff and the time.. then yes, a custom MTA solution is the best. But honestly.. most sites don't have the resources to put into that.

 

[ActorMike]

most sites don't have the resources to put into that.

It's really about knowledge, not resources; just like folks set up their own website and forum, if you know how to set up your own server, it's not hard to do it yourself.

I will let everyone know how they respond to my latest increase request. Basically, I gave them an ultimatum, which makes sense considering our own server is 5 times faster and the IP address has zero blacklists on it.

Now I see why Amazon offers the 30.00/mo dedicated IP address, because theirs are blacklisted.

 

[Ricsca]

I requested registration with Amazon SeS but Amazon replied that it won't activate it for my forum.
How is it possible?

 

[ActorMike]

I requested registration with Amazon SeS but Amazon replied that it won't activate it for my forum.
How is it possible?

What type of forum?

 

[TPerry]

It's really about knowledge, not resources; just like folks set up their own website and forum, if you know how to set up your own server, it's not hard to do it yourself.

It's about both... just because you have the knowledge, you may not have the resources to devote to a standalone MTA offering. I have the knowledge easily.. but I really would not prefer to have to expend the resources when I can meet the needs by SES. Even with the fact that the IP is on one block list, the mail still gets successfully delivered to 100% of the recipients.

Now I see why Amazon offers the 30.00/mo dedicated IP address, because theirs are blacklisted.

Explain how that's any different than me needing to go out and get a separate server instance ($30-$50 in value) with a dedicated IP and then me needing to go through the process of making sure the IP is clean and then maintaining that MTA instance. In the long run, the dedicated SES IP will generally be cheaper when you take into consideration administration costs.
It's something that those that are familiar with the process (which I count myself as one) seem to overlook when taking into consideration the general site owner ability.

 

[TPerry]

I requested registration with Amazon SeS but Amazon replied that it won't activate it for my forum.
How is it possible?

Apparently you did not provide an adequate description to them of your use of their services and what processes you took to prevent bounced/trash email.

Simply saying something like "I'm sending emails from my website" isn't going to cut the mustard.