What is your self-hosted SMTP solution? We are testing Postal at the moment, with speed issues

Helmuts

Helmuts

New member
Good morning,

Question to large and old forum owners whose email bounce-rates are above 10-15% - what SMTP server/vps setup do you use?

Last year I aquired a 530k member forum (est. in 2001) and 15.5k member forum (est. 2004). Note: both forums serve a particular niche and almost all the established industry pros have an account at either of these.

For those who suggest cleaning the accounts > deleting dead email accounts and "0" accounts is not smart. Ocassionally an industry leader or pro returns to one of the forums and asks ro check his old account and update it with the correct email or passwords. Or, a "0" comment account might send you a DM asking something (just happened to me this morning).

The aim is to set up own SMTP server to be able to send out system emails with a rate 20-100 emails/minute, that also provides a visual dashboard.

The issue with all the 3rd party smtp solutions (apart from costs) > 10-20% bounce-rate will result in a suspended accounts >> then you need to write the support, explain the issue, promise not to repeat the process and agree that the rest of pending outgoing emails are deleted. Result: emails have been sent out only to a part of the community, and you can't restart it as duplicate messages might trigger spam complaints an/or unsubs from newsletters.

At the moment we are testing Postal with the largest of forums.
The sever is overkill by all means: 64gb ram, 6c/12t



:) We are posting questions at the Postal's discord (as I write this) in order to get the solution. Issue is: outgoing message rate is 2 messages/second, and the server's load is minimal.

--

So, the question is: what is your self-hosted SMTP solution? Exim, Haraka, Docker Mailserver, Postfix, or have you managed to configure Postal that is works in acceptable speeds?

Thank you

Great day all!
H

Screenshot:

330k+ queued messages
Screenshot 2024-04-30 at 11.53.08.webp

htop shows that the server is barelly loaded:
Screenshot 2024-04-30 at 11.51.40.webp
 
Helmuts said:
For those who suggest cleaning the accounts > deleting dead email accounts and "0" accounts is not smart. Ocassionally an industry leader or pro returns to one of the forums and asks ro check his old account and update it with the correct email or passwords. Or, a "0" comment account might send you a DM asking something (just happened to me this morning).
Click to expand...
Why are you sending emails to these sorts of accounts?
If they are 'dead' and causing bounces, then why wouldn't you put the emails through a cleanse/verification service? Eg. https://www.millionverifier.com/ would do your 500k for under $300 USD.
 
Mouth said:
Why are you sending emails to these sorts of accounts?
If they are 'dead' and causing bounces, then why wouldn't you put the emails through a cleanse/verification service? Eg. https://www.millionverifier.com/ would do your 500k for under $300 USD.
Click to expand...

fly said:
Agreed. A 10-15% bounce rate isn't even polite. Get the list cleaned and then you can use a normal mail service like AWS SES.
Click to expand...

You are both correct. We stopped sending out the pending notifications after valuating the high bounce-rate.

Will be diving into XF foru and reading about the best XF cleaning practices and experiences.

Best wishes all!
H
 
Hey this leads me to a question. Is there an easy way to using a list cleaning service, then import the flagged accounts back into XF? TBH, I haven't looked into this at all.
 
fly said:
Hey this leads me to a question. Is there an easy way to using a list cleaning service, then import the flagged accounts back into XF? TBH, I haven't looked into this at all.
Click to expand...
And now I see a post from @eva2000 suggesting he does it somewhat manually.
ProCom

Thread 'Email Newsletter List Bounce Cleanup / Scrubber Services'

I recently converted a large forum to XF. When I run the "Email Users" script, the list of valid email addresses that I get from XF is 350,000. Unfortunately the list that my old provider was sending through SendGrid was only 270,000. That 80k difference is a concern to me and I'm worried that the data that I got from my previous provider didn't have all the proper bounce details.

I'm worried that my imported list isn't as clean as the provider said it would be, so I decided to run my "0 post members" (about 150k) through a list scrubber to reduce the potential bounces.

As a test, I...

edit: But it looks like @DragonByte Tech has an addon that will do it for you.
 
Last edited:
fly said:
now I see a post from @eva2000 suggesting he does it somewhat manually.
Click to expand...
Yeah I do it manually

edit: actually decided to try using python libraries that can do similar checks to DragonByte Tech's Mail PHP library and looking good so far, script can handle individual emails or files with emails

email syntax/domain/domain mx validation works at least

With disposable email domain database support

Code: 
python validate_emails.py -f user@domain.com -e user+to@domain.com,user@tempr.email
[
    {
        "email": "user+to@domain.com",
        "status": "ok"
    },
    {
        "email": "user@tempr.email",
        "status": "disposable"
    }
]

Code: 
python validate_emails.py -f username@domain.com -e username+to@domaininvalid.com
2024-05-02 01:36:18,902 - WARNING - Validation failed for username+to@domaininvalid.com: The domain name domaininvalid.com does not exist.
[
    {
        "email": "username+to@domaininvalid.com",
        "status": "invalid_format"
    }
]
Code: 
python validate_emails.py -f user@domain.com -e user+to@domainwithinvalidmx.com
2024-05-02 01:58:19,277 - INFO - Validating user+to@domainwithinvalidmx.com
2024-05-02 01:58:19,277 - ERROR - DNS resolution failed for domainwithinvalidmx.com: The DNS response does not contain an answer to the question: domainwithinvalidmx.com. IN MX
[
    {
        "email": "user+to@domainwithinvalidmx.com",
        "status": "dns_resolution_error"
    }
]
syntax only test without smtp and dns test
Code: 
python validate_emails.py -f user@domain.com -e user+to@domain.com -tm syntax
2024-05-02 02:23:58,489 - INFO - Validating user+to@domain.com
[
    {
        "email": "user+to@domain.com",
        "status": "valid_format"
    }
]

syntax and dns tests only without smtp test

Code: 
python validate_emails.py -f user@domain.com -e user+to@domain.com -tm dns
2024-05-02 02:24:46,193 - INFO - Validating user+to@domain.com
[
    {
        "email": "user+to@domain.com",
        "status": "valid_dns"
    }
]

fly said:
But it looks like @DragonByte Tech has an addon that will do it for you
Click to expand...
Which addon is that ? Ah Mail addon
 
Last edited:
eva2000 said:
Yeah I do it manually

edit: actually decided to try using python libraries that can do similar checks to DragonByte Tech's Mail PHP library and looking good so far, script can handle individual emails or files with emails

email syntax/domain/domain mx validation works at least

With disposable email domain database support

Code: 
python validate_emails.py -f user@domain.com -e user+to@domain.com,user@tempr.email
[
    {
        "email": "user+to@domain.com",
        "status": "ok"
    },
    {
        "email": "user@tempr.email",
        "status": "disposable"
    }
]

Code: 
python validate_emails.py -f username@domain.com -e username+to@domaininvalid.com
2024-05-02 01:36:18,902 - WARNING - Validation failed for username+to@domaininvalid.com: The domain name domaininvalid.com does not exist.
[
    {
        "email": "username+to@domaininvalid.com",
        "status": "invalid_format"
    }
]
Code: 
python validate_emails.py -f user@domain.com -e user+to@domainwithinvalidmx.com
2024-05-02 01:58:19,277 - INFO - Validating user+to@domainwithinvalidmx.com
2024-05-02 01:58:19,277 - ERROR - DNS resolution failed for domainwithinvalidmx.com: The DNS response does not contain an answer to the question: domainwithinvalidmx.com. IN MX
[
    {
        "email": "user+to@domainwithinvalidmx.com",
        "status": "dns_resolution_error"
    }
]
syntax only test without smtp and dns test
Code: 
python validate_emails.py -f user@domain.com -e user+to@domain.com -tm syntax
2024-05-02 02:23:58,489 - INFO - Validating user+to@domain.com
[
    {
        "email": "user+to@domain.com",
        "status": "valid_format"
    }
]

syntax and dns tests only without smtp test

Code: 
python validate_emails.py -f user@domain.com -e user+to@domain.com -tm dns
2024-05-02 02:24:46,193 - INFO - Validating user+to@domain.com
[
    {
        "email": "user+to@domain.com",
        "status": "valid_dns"
    }
]


Which addon is that ? Ah Mail addon
Click to expand...

Hahhh.. nice :) We will also look into this. Thank you for sharing.
 
Helmuts said:
Hahhh.. nice :) We will also look into this. Thank you for sharing.
Click to expand...
Yeah, the risk of doing it yourself is potentially damaging your sending email domain's reputation on the server you run the script from ! Paid email cleaning services as mechanisms in place to combat this i.e. monitoring feedback loops, closer working relationship with ISP/email providers and dedicated IPs/IP rotations.

Logging SMTP server response displayed in -v verbose mode with email addresses in file emaillist.txt

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -v
2024-05-02 07:52:19,212 - INFO - Disposable email address: user@tempr.email
2024-05-02 07:52:20,213 - INFO - Validating user+to@domain1.com
2024-05-02 07:52:20,213 - INFO - Validating xyz@domain1.com
2024-05-02 07:52:20,317 - INFO - Validating user@domain2.com
2024-05-02 07:52:20,843 - INFO - SMTP response for xyz@domain1.com: 550, b"5.1.1 The email account that you tried to reach does not exist. Please try\n5.1.1 double-checking the recipient's email address for typos or\n5.1.1 unnecessary spaces. For more information, go to\n5.1.1  https://support.google.com/mail/?p=NoSuchUser l21-20020aa7d955000000b00572b128de80si241174low.690 - gsmtp"
2024-05-02 07:52:20,900 - INFO - SMTP response for user@domain2.com: 250, b'2.1.5 OK f16-20020a5d6650000000b0034d76344471si303235abc.904 - gsmtp'
2024-05-02 07:52:20,912 - INFO - SMTP response for user+to@domain1.com: 250, b'2.1.5 OK h21-20020aa7c615000000b0056e71418614si244037xzy.505 - gsmtp'
[
    {
        "email": "user+to@domain1.com",
        "status": "ok"
    },
    {
        "email": "xyz@domain1.com",
        "status": "unknown_email"
    },
    {
        "email": "user@tempr.email",
        "status": "disposable"
    },
    {
        "email": "user@domain2.com",
        "status": "ok"
    }
]

Though I added a DNS mode that skips SMTP checks so you can reduce the number of bad emails down and filter out invalid DNS before doing actual SMTP checks :)

Code: 
python validate_emails.py -f user@domain1.com -l emaillist.txt -tm dns -v
2024-05-02 08:05:31,142 - INFO - Disposable email address: user@tempr.email
2024-05-02 08:05:32,142 - INFO - Validating xyz@domain1.com
2024-05-02 08:05:32,142 - INFO - Email DNS is valid: xyz@domain1.com
2024-05-02 08:05:32,143 - INFO - Validating user+to@domain1.com
2024-05-02 08:05:32,143 - INFO - Email DNS is valid: user+to@domain1.com
2024-05-02 08:05:32,156 - INFO - Validating user@domain2.com
2024-05-02 08:05:32,156 - INFO - Email DNS is valid: user@domain2.com
[
    {
        "email": "user+to@domain1.com",
        "status": "valid_dns"
    },
    {
        "email": "xyz@domain1.com",
        "status": "valid_dns"
    },
    {
        "email": "user@tempr.email",
        "status": "disposable"
    },
    {
        "email": "user@domain2.com",
        "status": "valid_dns"
    }
]
 
Last edited:
eva2000 said:
Yeah, the risk of doing it yourself is potentially damaging your sending email domain's reputation on the server you run the script from ! Paid email cleaning services as mechanisms in place to combat this i.e. monitoring feedback loops, closer working relationship with ISP/email providers and dedicated IPs/IP rotations.
Click to expand...
Since that comment, my email verification script now supports 5 commercial email verification services' APIs for - EmailListVerify, MillionVerifier, MyEmailVerifier, CaptainVerify, Proofy.io . Comparison tables for pricing and results here and full details and demo examples at https://github.com/centminmod/validate-emails. So no risk of reputation damage :D

For @Helmuts to compare pricing for >500k

Provider200k250k300k500k1m2.5m5m10m
EmailListVerify (demo, results)-$349 (0.001396)-$449 (0.000898)$599 (0.000599)$1190 (0.000476)$1990 (0.000398)$3290 (0.000329)
MillionVerifier (demo, results)---$259 (0.000518)$389 (0.000389)-$1439 (0.000288)$2529 (0.000253)
MyEmailVerifier (demo, results)-$349 (0.001396)-$549 (0.001098)$749 (0.000749)$1249 (0.0005)$1849 (0.00037)-
CaptainVerify (demo, results)-$250 (0.001)-$500 (0.001)$650 (0.00065)-$2000 (0.0004)-
Proofy.io (demo, results)$229 (0.001145)-$289 (0.000963)$429 (0.000858)$699 (0.000699)$1399 (0.00056)--
 
Excerpt from larger comparison table showing how each commercial service handles a disposable email that is work, an invalid domain, a valid working email and a user account that doesn't exist versus local script lookup result.

Table comparing the JSON field values for each email address across the 5 different Email cleaning service APIs and also compared to local script non-API queries results.
Tested on the same sample emaillist.txt of email addresses. These are their respective returned values for status JSON field which retrieved from the respective API services. While status_code (not used with external APIs), free_email and disposable_email JSON fields are from local script code/databases where applicable.

EmailAPIstatusstatus_codefree_emaildisposable_email
user@mailsac.comEmailListVerifyunknownnullyesyes
user@mailsac.comMillionVerifierdisposablenullfalseyes
user@mailsac.comCaptainVerifyriskynullnoyes
user@mailsac.comProofy.ioundeliverablenullnoyes
user@mailsac.comMyEmailVerifierinvalidnullyesyes
user@mailsac.comLocal Scriptok250yesyes
xyz@centmil1.comEmailListVerifyunknownnullnono
xyz@centmil1.comMillionVerifierinvalidnullfalseno
xyz@centmil1.comCaptainVerifyinvalidnullnono
xyz@centmil1.comProofy.ioundeliverablenullnono
xyz@centmil1.comMyEmailVerifierinvalidnullnono
xyz@centmil1.comLocal Scriptinvalid_formatnullunknownno
user+to@domain1.comEmailListVerifyvalidnullnono
user+to@domain1.comMillionVerifieroknullfalseno
user+to@domain1.comCaptainVerifyvalidnullnono
user+to@domain1.comProofy.iodeliverablenullnono
user+to@domain1.comMyEmailVerifiervalidnullnono
user+to@domain1.comLocal Scriptok250nono
xyz@domain1.comEmailListVerifyemail_disablednullnono
xyz@domain1.comMillionVerifierinvalidnullfalseno
xyz@domain1.comCaptainVerifyinvalidnullnono
xyz@domain1.comProofy.ioundeliverablenullnono
xyz@domain1.comMyEmailVerifierinvalidnullnono
xyz@domain1.comLocal Scriptunknown_email550nono
 
Also, commercial email verification providers usually only store your file-based uploaded or bulk file API uploaded files for a defined duration, i.e. 30 days before they are deleted. And per email check API results are usually not stored at all. So if you need to store your per email check or bulk file API email verification results for longer, my validate_emails.py script now supports saving your results to S3 object storage providers - Cloudflare R2 or Amazon AWS S3 :D

example

Send validate_emails.py script results to Cloudflare R2 S3 object storage via -store r2 argument. Using EmailListVerify per email check API -api emaillistverify -apikey $elvkey + Cloudflare cached for 120 seconds -apicache emaillistverify -apicachettl 120

Bash: 
time python validate_emails.py -f user@domain1.com -e hnyfmw@canadlan-drugs.com,hnyfmw2@canadlan-drugs.com,hnyfmw3@canadlan-drugs.com -api emaillistverify -apikey $elvkey -apicache emaillistverify -apicachettl 120 -tm all -store r2

Output stored successfully in R2: emailapi-emaillistverify-cached/output_20240511051940.json
[
    {
        "email": "hnyfmw@canadlan-drugs.com",
        "status": "unknown",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "hnyfmw2@canadlan-drugs.com",
        "status": "unknown",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no"
    },
    {
        "email": "hnyfmw3@canadlan-drugs.com",
        "status": "unknown",
        "status_code": null,
        "free_email": "no",
        "disposable_email": "no"
    }
]

real    0m1.663s
user    0m0.391s
sys     0m0.039s
 
Helmuts said:
Good morning,

Question to large and old forum owners whose email bounce-rates are above 10-15% - what SMTP server/vps setup do you use?

Last year I aquired a 530k member forum (est. in 2001) and 15.5k member forum (est. 2004). Note: both forums serve a particular niche and almost all the established industry pros have an account at either of these.

For those who suggest cleaning the accounts > deleting dead email accounts and "0" accounts is not smart. Ocassionally an industry leader or pro returns to one of the forums and asks ro check his old account and update it with the correct email or passwords. Or, a "0" comment account might send you a DM asking something (just happened to me this morning).

The aim is to set up own SMTP server to be able to send out system emails with a rate 20-100 emails/minute, that also provides a visual dashboard.

The issue with all the 3rd party smtp solutions (apart from costs) > 10-20% bounce-rate will result in a suspended accounts >> then you need to write the support, explain the issue, promise not to repeat the process and agree that the rest of pending outgoing emails are deleted. Result: emails have been sent out only to a part of the community, and you can't restart it as duplicate messages might trigger spam complaints an/or unsubs from newsletters.

At the moment we are testing Postal with the largest of forums.
The sever is overkill by all means: 64gb ram, 6c/12t



:) We are posting questions at the Postal's discord (as I write this) in order to get the solution. Issue is: outgoing message rate is 2 messages/second, and the server's load is minimal.

--

So, the question is: what is your self-hosted SMTP solution? Exim, Haraka, Docker Mailserver, Postfix, or have you managed to configure Postal that is works in acceptable speeds?

Thank you

Great day all!
H

Screenshot:

330k+ queued messages
View attachment 302039

htop shows that the server is barelly loaded:
View attachment 302038
Click to expand...

Since no one else pointed it out, I'm going to point out and say even if you did run your own SMTP server, you wouldnt get far because the reason you get suspended at 15-20% bounces is because the mail senders have a reputation to uphold. If you have a high bounce rate running an SMTP server its game over, your IP will get blacklisted and all your outgoing emails will be dropped or marked junk even the non-bouncing ones. STMP server reputation is very fragile.
 
Good info in this thread.

After acquiring another forum, and merging a percentage of it's content into my own, I used a email cleaning service to audit my entire combined user profile emails. Disabled all of the invalid email/profiles with posts, and deleted the invalid email/profiles with no posts or uploaded images. Use Andy's addon to delete new profiles that never validate or post something.

I use Postmark as my outbound SMTP Service. They offer some management services to alert you on the items that would want to be aware of. A bit pricer than other options, but manage very few issues myself.
 
We run Mailcow and relay all outgoing e-mails via AWS SES, which costs only a few cents every month.
 
I've been thinking about this since Amazon SES is the cheapest but can still get costly if you're sending out lots of emails. A self-hosted SMTP should be much cheaper and if you start off with a clean list you shouldn't have to worry about email reputation, or even landing in spam, for forum notifications.

I haven't tried a self-hosted SMTP yet but I know that the listmonk.app guy sends millions of emails with Haraka SMTP at ~50k/minute. So you shouldn't experience that low Postal rate issue.

I actually found this thread when I was thinking about asking the Xenforo devs why they use Amazon SES for this forum instead of a self-hosted SMTP.
 
eva2000 said:
Since that comment, my email verification script now supports 5 commercial email verification services' APIs for - EmailListVerify, MillionVerifier, MyEmailVerifier, CaptainVerify, Proofy.io . Comparison tables for pricing and results here and full details and demo examples at https://github.com/centminmod/validate-emails. So no risk of reputation damage :D

For @Helmuts to compare pricing for >500k

Provider200k250k300k500k1m2.5m5m10m
EmailListVerify (demo, results)-$349 (0.001396)-$449 (0.000898)$599 (0.000599)$1190 (0.000476)$1990 (0.000398)$3290 (0.000329)
MillionVerifier (demo, results)---$259 (0.000518)$389 (0.000389)-$1439 (0.000288)$2529 (0.000253)
MyEmailVerifier (demo, results)-$349 (0.001396)-$549 (0.001098)$749 (0.000749)$1249 (0.0005)$1849 (0.00037)-
CaptainVerify (demo, results)-$250 (0.001)-$500 (0.001)$650 (0.00065)-$2000 (0.0004)-
Proofy.io (demo, results)$229 (0.001145)-$289 (0.000963)$429 (0.000858)$699 (0.000699)$1399 (0.00056)--
Click to expand...

Just for my own benefit, here are the prices for SparkPost's recipient validation service:
  • 200K $600 ($0.003 per address)
  • 250K $375 ($0.0015 per address)
  • 300K $450 ($0.0015 per address)
  • 500K $750 ($0.0015 per address)
  • 1m $750 ($0.00075 per address)
  • 2.5m $1,875 ($0.00075 per address)
  • 5m $3,750 ($0.00075 per address)
  • 10m $7,500 ($.00075 per address)
... SparkPost's pricing stops at 1m addresses, so I wonder if you had a larger list to be cleansed, whether they would do better pricing.

Either way, SparkPost are pretty expensive for smaller lists compared to some of the above. Lists are only kept for 10 days, so must be downloaded after validation.

Here's an example of what the JSON payload from the API returns for a single address query:
JSON: 
  {
      "results": {
          "result": "undeliverable",
          "valid": false,
          "reason": "Invalid Domain"
           "is_role": false,
          "is_disposable": false,
          "delivery_confidence": 0,
          "is_free": false
      }
  }
 
MaximilianKohler said:
I've been thinking about this since Amazon SES is the cheapest but can still get costly if you're sending out lots of emails. A self-hosted SMTP should be much cheaper and if you start off with a clean list you shouldn't have to worry about email reputation, or even landing in spam, for forum notifications.

I haven't tried a self-hosted SMTP yet but I know that the listmonk.app guy sends millions of emails with Haraka SMTP at ~50k/minute. So you shouldn't experience that low Postal rate issue.

I actually found this thread when I was thinking about asking the Xenforo devs why they use Amazon SES for this forum instead of a self-hosted SMTP.
Click to expand...
Do you really have a (non spam) usecase where SES gets 'expensive'? You can send approximately 66,000 emails a day for $225/mo.
 
You must log in or register to reply here.
Back
Top Bottom