XF 2.0 TLS Mailhandling

[Sperber]

Most of us will have a server with one mailserver and run their mails of all domains through this single mailserver. For TLS encryption the mailserver will be setup with a certifcate that contains a unique domain name which the certificate is issued to ["CN"]. When trying to send mails with TLS encryption from the same server but a different domain name, the mailhandler in XenForo rejects the certificate f the mailserver as a hard fail and sending is only possible with TLS deactivated.

Since we all would like our forums to send TLS encrypted mails, for webmasters there should be some sort of manaul approval or another workaround, so the mailhandler accepts the mailserver certificate, despite the different CN domain name in the certificate. Same way, as our desktop mailagents handle that.

 

[Mike]

If I'm understanding what you're reporting, you're saying that you're connecting to domain.com as your mail server, but your mail server is using a certificate issued for somethingelse.com.

If so, then I don't really see why we should allow this, as you would effectively have to turn off certificate validation to do this (partially defeating the point of certs). Meanwhile, shouldn't you just be able to connect to somethingelse.com (on the assumption that your mail server certificate is actually for a domain it resolves to)?

 

[Sperber]

If I'm understanding what you're reporting, you're saying that you're connecting to domain.com as your mail server, but your mail server is using a certificate issued for somethingelse.com.

If so, then I don't really see why we should allow this, as you would effectively have to turn off certificate validation to do this (partially defeating the point of certs).

Insofar correct, Mike. My point is: when domain.com is the mailserver and somethingelse.com is configured to send mails through the mailserver at domain.com, then this is a pretty common situation. Some only have one mailserver available - or just don´t want to configure a seperate one for each domain. Regarding the validation chain, the mails send this way are 100% validated, as long as the DNS records on somethingelse.com are setup properly and it´s complying with the RFCs.

 

[Mike]

What is the actual error and backtrace that you're getting?

 

[Sperber]

That the mailserver CN in the certificate is different from the domain where your forum is running. It is handled as a hard fail in swift, so swift terminates the connection. If this could be set to a softfail - so you have the option to say accept the certificate via a setting that best would be saved in a config - this should probably solve this.

 

[Sim]

Given that this is exactly how ESPs like SparkPost work, I'm not sure why you are having a problem?

SparkPost has a mail server. I can send emails to it using TLS over SMTP, it then sends the emails for me. Everyone else connects to the same mail server too, to send emails from their domain.

Seems like you have a mail server configuration problem to me?

 

[Sperber]

Given that this is exactly how ESPs like SparkPost work, I'm not sure why you are having a problem?

SparkPost has a mail server. I can send emails to it using TLS over SMTP, it then sends the emails for me. Everyone else connects to the same mail server too, to send emails from their domain.

Seems like you have a mail server configuration problem to me?

No. Even when I don´t know SparkPost and I don´t know how their mails validate in an mx analysis. May be they have an own CA or a commercial one that automaticly adds customer domains to the certificate SparkPost uses on his mailservers. But that doesn´t work with Let´s encrypt or mid priced CA´s issuing single domain certs. Of course everyone can buy a multidomain cert for a thousand bugs and of course this will solve the described problem. I guess the people who can afford and go this way don´t pass the 1% line and instead want to use their LE or the certs they already have.

 

[Mike]

The domain of your forum shouldn't have anything to do with the certificate validation of your mail server. If you tell PHP to connect to domain.com (when making the mail server connection), then it will expect the certificate to say it's for domain.com -- this is an important part of certificate validation (that you are talking to who you think you are, based on authorities you trust). If it says it's for mailserver.com, then it's reasonable to reject that because that isn't the server your expecting. In that case, you should just connect to mailserver.com directly. (If that's not possible for some reason, then your mail server provider really needs to sort that, as otherwise they are essentially forcing you to bypass certificate validation and that undoes a chunk of the security it provides.)

 

[Sim]

No. Even when I don´t know SparkPost and I don´t know how their mails validate in an mx analysis. May be they have an own CA or a commercial one that automaticly adds customer domains to the certificate SparkPost uses on his mailservers. But that doesn´t work with Let´s encrypt or mid priced CA´s issuing single domain certs. Of course everyone can buy a multidomain cert for a thousand bugs and of course this will solve the described problem. I guess the people who can afford and go this way don´t pass the 1% line and instead want to use their LE or the certs they already have.

Nope - they don't add customer domains to the certificate - and they don't need to. Their mail server is the one sending the emails - not my domain.

The domain you are sending from has nothing to do with the certificate of the mail server.

I can send mail from a local dev sever (where it is impossible to get an SSL certificate because not using a routable domain) via SparkPost using TLS - the TLS bit is only about point-to-point communication and not about email headers or validation.

My sending email address is from a valid domain though - that's what they validate when sending, although again, that has nothing to do with TLS.

TLS (in a mail context) is about mail servers (or applications) talking to mail servers and not about email addresses or websites.

 

[Kirby]

This Bug report confuses the hell out of me.

We do have a TLS capable mailserver at mail.company.com which only has a LetsEncrypt certificate for that domain.
Yet we use it to send emails for our forums running on multiple different domains though this server without issues.

So I really do not understand the problem here?

 

[Sperber]

This Bug report confuses the hell out of me.

In the meantime it does for me either, Kirby, since this would indicate my servers run mailservers in a way, they shouldn´t be able to operate

Each of my mailserver serves several domains on the same IP. They work like described and the mails from all domains validate 10/10. We never had any problems with that and even in XF 1.5 it has and still is working flawlessly with TLS enabled. The problem arised - for me - with XF 2:

ErrorException: Email to xxx@domain.com failed: [E_WARNING] stream_socket_enable_crypto(): Peer certificate CN=`mailserverdomain.com' did not match expected CN=`forumsdomain.com' src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php:103

Stack trace

#0 [internal function]: XF::handlePhpError(2, '[E_WARNING] str...', '/var/www/vhosts...', 103, Array)
#1 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php(103): stream_socket_enable_crypto(Resource id #203, true, 57)
#2 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/EsmtpTransport.php(313): Swift_Transport_StreamBuffer->startTLS()
#3 src/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(118): Swift_Transport_EsmtpTransport->_doHeloCommand()
#4 src/XF/Mail/Mailer.php(274): Swift_Transport_AbstractSmtpTransport->start()
#5 src/XF/Mail/Mail.php(388): XF\Mail\Mailer->send(Object(Swift_Message), Object(Swift_SmtpTransport), NULL, false)
#6 src/XF/Admin/Controller/Tools.php(276): XF\Mail\Mail->send(Object(Swift_SmtpTransport), false)
#7 src/XF/Mvc/Dispatcher.php(321): XF\Admin\Controller\Tools->actionTestEmail(Object(XF\Mvc\ParameterBag))
#8 src/XF/Mvc/Dispatcher.php(248): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'TestEmail', Object(XF\Mvc\RouteMatch), Object(xenMade\ACPE\XF\Admin\Controller\Tools), NULL)
#9 src/XF/Mvc/Dispatcher.php(100): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(xenMade\ACPE\XF\Admin\Controller\Tools), NULL)
#10 src/XF/Mvc/Dispatcher.php(50): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#11 src/XF/App.php(2177): XF\Mvc\Dispatcher->run()
#12 src/XF.php(392): XF\App->run()
#13 admin.php(13): XF::runApp('XF\\Admin\\App')
#14 {main}

The domain names in the error report are those from the corresponding DNS records.

We do have a TLS capable mailserver at mail.company.com which only has a LetsEncrypt certificate for that domain.
Yet we use it to send emails for our forums running on multiple different domains though this server without issues.
So I really do not understand the problem here?

Are your forum mails send with TLS enabled in XF 2?

 

[Kirby]

Are your forum mails send with TLS enabled in XF 2?

Normally this is not the case (we do have forwarders running on localhost of each node, XF connects to those local postfix instances which will then queue mail to the real mailserver), but I've reconfigured a test forum just for you:

++ Starting Swift_SmtpTransport
<< 220 mail.company.com ESMTP

>> EHLO forumdomain.com

<< 250-mail.company.com
250-PIPELINING
250-SIZE 51200000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

>> STARTTLS

<< 220 2.0.0 Ready to start TLS

>> EHLO forumdomain.com

<< 250-mail.company.com
250-PIPELINING
250-SIZE 51200000
250-VRFY
250-ETRN
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

++ Swift_SmtpTransport started
>> MAIL FROM:<info@forumdomain.com>

<< 250 2.1.0 Ok

>> RCPT TO:<my@private-email-address.com>

<< 250 2.1.5 Ok

>> DATA

<< 354 End data with <CR><LF>.<CR><LF>

>>
.

<< 250 2.0.0 Ok: queued as 3C11E63323

As you can see, this works just fine (XF 2.1 Beta 5).

Normally all you have to do is connect to the mailserver with a hostname that has a certificate, eg. mailserverdomain.com

 

[Sperber]

Normally all you have to do is connect to the mailserver with a hostname that has a certificate, eg. mailserverdomain.com

Of course, that´s how it´s setup. I am using roundcube, but that shouldn´t make a real difference. Thanks for testing this in your testbed - even when it´s giving me some headaches now. Mike, as this seems to be not a bug, could you please move this thread to somewhere appropriate. Guess this is something that I should indeed elaborate further on.

 

[Sim]

Stuff like this is one of the reasons I stopped running my own mail servers

 

[Kirby]

Yeah, maintaining mailservers is not thaat funny, but sending some million mails each month through mail service providers is quite expensive and legally troublesome

 

[Mike]

We never had any problems with that and even in XF 1.5 it has and still is working flawlessly with TLS enabled.

XF 1.x generally didn't do CN validation, and that was a major failing with it. We've taken steps to resolve some of the issues that make this hard to do (such as by shipping the necessary base certificates) so that 2.x can do it out of the box.

Of course, that´s how it´s setup.

If you can send me a screenshot of your mail configuration in XF (I don't need the password as the TLS setup happens before that), then I could at least try to mimic the configuration and see if I can reproduce the issue or give you a specific recommendation.

 

[Kirby]

XF 1.x generally didn't do CN validation

That might very well be the reason why "it does work" in XF 1.5