Mail Sending Limits - What are large sites doing?

MapleOne

Member
So most webhosts limit the amount of emails one can send from a hosting package.

So what are Xenforo owners with really large forums doing for email?

So lets say you have 300 outgoing emails notifications for the hour and the host kills them at 100.

I know Xenforo has the built in notification but on really large forums when a ton of members subscribe to email notifications you are bound to go over the send limits. Then you have the members that quit the forum and instead of unsubscribing they mark you as spam and eventually the IP get flagged.

I am looking for advice from members with larger forums to see how they handle this mail issue with send limits.

I own a couple od smaller forums and am already running into this issue.
 

Sim

Well-known member
Never ever use your web host's SMTP servers for sending email, even for a small site - your deliverability rates will be very low.

As a way to combat spam, email providers have become very strict in monitoring the sending reputation of IP addresses - there are plenty of people out there sending large volumes of low quality email (or even spam) from their websites and if you're using the same shared SMTP server, then your reputation will be affected as well.

This means that emails you send won't reach the inbox of your users.

A 3rd party email service provider (ESP) like AWS SES, Sendgrid, SparkPost, etc is going to get you a much better result and hugely improve your deliverability.

Large forums will use an ESP and have their own dedicated IP address for sending emails - they also carefully manage their own sending reputation to ensure that they maintain high levels of deliverability.

To maintain high sending reputation you must keep your sending list clean - which means processing bounced emails in XenForo to avoid sending to uses whose accounts are invalid or who have blocked you.

I'm currently sending around 80,000 emails per month from my sites (8 forums, about 20 small websites and numerous other web applications that I host) - I use SparkPost and have my own dedicated IP address from them for sending email. It's not cheap: US$75 per month which includes a dedicated IP - but my deliverability rates are very high.
 

Wildcat Media

Well-known member
For one site I've been an admin for since 2004, which is still at the original host, we've used our server's outgoing SMTP and have never had an issue. When I had other sites hosted there, as far back as 1997, no issues with those either.

That said, if we have to move to another host in the future, I am prepared to move mail hosting elsewhere. For some smaller projects, I host them at DigitalOcean, and their outgoing mail is notorious for being flagged as spam, since it is too easy for spammers to sign up for accounts there.

I had to reluctantly use SES at Amazon (I prefer not giving Amazon my business)--getting accepted to use an SES account is a hassle, but it worked, and mail is being delivered.
 

Digital Doctor

Well-known member
This means that emails you send won't reach the inbox of your users.

would these things help ?

Have you set SPF, DKIM, DMARC, and PTR (Reverse Pointer) records for email on your server?

Actually. the link posted by @duderuud will tell you about DKIM and SPF but you still need to to check DMARC and PTR.

Hotmail and others block my email routinely
 

sixlxvi

Well-known member
I use Amazon SES to send about 8,000 emails a month in a newsletter, plus however many are sent from forum alerts and such. It’s very cheap and has a great deliverability rate for me. It can be confusing to set up but Matt W on this forum offers a great configuration service (https://mattwservices.co.uk/).

I have been curious about something else though. Does anyone know anything about sending limits with Apple’s new iCloud Mail custom domain system?
 

Sim

Well-known member
would these things help ?

You should always set up SPF and DKIM.

PTR requires you have exclusive use of the IP address because there needs to be a 1:1 mapping of IP address to hostname. Usually your hosting provider will take care of this for you though. It only becomes an issue if the PTR needs to resolve to a domain you own (rather than your hosting provider), which would only be the case in very specific circumstances.

DMARC is more difficult to set up and getting it wrong can break your mail deliverability. You'll also need to set up a system for analysing the DMARC reports you'll be sent - most of which are in XML format and not intended to be human-readable.

An ESP will help with setup of all of these - for example, SparkPost make it trivial to set up SPF and DKIM, they take care of the PTR for you and have tutorials on how to set up DMARC.

I haven't been through the process of setting up DMARC for any of my domains yet - I haven't had to time to digest all of the information and understand exactly what the ramifications of each of the settings is, nor do I have the infrastructure in place to handle DMARC reports.
 

Wildcat Media

Well-known member
would these things help ?
I set all of those, and none of them helped on my DigitalOcean server. I was able to receive messages via Gmail, but I had members on other, major providers who would never see my forum emails (conversations, alerts, notifications, whatever...nothing got through).

And it does depend heavily on the hosting provider used, as I've found. Polar opposites, in my case.

Moved to SES and everyone is getting emails again. I just hate giving my money to Amazon...
 

Sim

Well-known member
I set all of those, and none of them helped on my DigitalOcean server

Sending IP address reputation matters more than anything - and SPF/DKIM/etc won't be able to fix it.

Trying to send emails from an IP address which is used by other people to send low quality email or spam, will definitely affect your deliverability.

Sending emails from a new IP address without an existing reputation is also difficult - you need to go through a "warmup" process to ensure maximum deliverability.
 

JOGARA

Member
Over 100k emails a month for me and I use AWS SES. No issues. Takes a few months to build up reputation though.

Never use the host email server for website notifications emails. Only for user accounts such as yourself and other administrators that may have email accounts.
 

arn

Well-known member
We use Sendgrid.com but it's not especially cheap. 200k emails in the last 30 days. In general, email deliverability is a nightmare and I'm happy not to have to deal with it too much, and outsourcing it helps with my sanity.
 

Sim

Well-known member
email deliverability is a nightmare

Yes, deliverability is probably the most challenging part of running any kind of website which sends emails to users these days - but fortunately there are ESPs like Sendgrid/SES/SparkPost/etc who specialise in this type of thing, I would hate to have to manage this stuff myself!
 

duderuud

Well-known member
For years I used own own mailserver but the hassle of keeping that server robust and up to date was just a bit too much.

Nowadays we use Amazon SES. Setting it up is also a bit of a hassle but after that it runs flawlessly.
 

Wildcat Media

Well-known member
Missed opportunity--I should have taken notes while I signed up for it. The only thing that held me up was that they wanted this long, involved description of what we would be using SES for. I had a brief description and they rejected it. Beyond that, it was just a matter of getting signed up for it and configuring everything in the right place. And I'm pretty sure they offered a way to copy and paste whatever we needed to use it in our own application (XF, in other words).
 
Top