XenForo accepts invalid email addresses

[Kirby]

Affected version

2.2.11

XenForo does accept email adresses like 1@1.1.

Such an email ist invalid because 1.1 ist not an IPv4 address, not an IPv6 address and not a valid FQDN:

RFC 3696: Application Techniques for Checking and Transformation of Names

There is an additional rule that essentially requires that top-level domain names not be all-numeric

PHP's built-in filter methods would correctly reject such an email:

var_dump(filter_var('1@1.1', FILTER_VALIDATE_EMAIL));

 

[PaulB]

PHP’s validation also rejects perfectly valid email addresses by modern standards, including IDNs in Unicode form. It will also fail to reject a variety of hostnames that aren’t valid by modern standards.

Hostname validation is extraordinarily complex and is constantly changing. It’s best to be permissive when validating emails and allow the MTA to handle formal validation. Either it can get the mail to its destination, or it can’t: that’s the only true test of validity.

There are domain names out in the wild that were valid when they were registered, but that are not considered valid by newer standards and cannot be re-registered once dropped. Many would be unable to get a valid TLS certificate from a public CA despite being valid public domains you can visit from any web browser.

 

[Xon]

It would be nice if XF checked that the domain part resolves to an A/AAAA/MX record, this would catch a number of invalid domains without needing to code explicit validation

 

[PaulB]

That would work, but it seems rather pointless given that the MTA is better equipped to handle that and is performing that check anyway.

 

[Kirby]

That would work, but it seems rather pointless given that the MTA is better equipped to handle that and is performing that check anyway.

I totally disagree here.

I came across this issue because on one of our installations XF erorr log was flooded with

Swift_TransportException: Email to 1@1.1 from 1@1.1 failed: Expected response code 354 but got code "554", with message "554 5.5.1 Error: no valid recipients "

Accepting invalid email adresses is problematic because it creates support workload, either for our ops team (in case of such errors being logged) or account managers (in case users accidently mistype an email address).

I'd therefore much rather prefer XenForo to reject some "edge case" emails as invalid (which most likely would happen seldom) than having users contact support after having registered with an invalid email address (which happens frequently).

 

[PaulB]

I came across this issue because on one of our installations XF erorr log was flooded with

That’s the real bug, though. XenForo should be able to handle that.

I'd therefore much rather prefer XenForo to reject some "edge case" emails as invalid (which most likely would happen seldom) than having users contact support after having registered with an invalid email address (which happens frequently).

The proposed solution rejects valid email addresses, though, which seems even worse. Are you requiring email confirmation during registration?

An email address ending with .1 is valid because the hostname portion of an email address can be an IP address—I’m not sure why someone would want to do that, but it could technically be a valid delivery target.

 

[Kirby]

The proposed solution rejects valid email addresses, though, which seems even worse. Are you requiring email confirmation during registration?

Which "proposed solution" are you refering to? I didn't propose a solution and the suggestion made by @Xon shouldn't reject valid email addresses (unless there is a (temporary) failure in DNS resolution or under "obscure" setups were PHP can't resolve but the SMTP server can).

As said before, I'd rather prefer such "false rejections" instead of users registering with invalid emails (due to typos).
Dealing with such edge cases would most likely be less work then dealing with users who mistyped their email address.

An email address ending with .1 is valid because the hostname portion of an email address can be an IP address [...] but it could technically be a valid delivery target.

An email address ending with .1 might be valid if the domain part is a valid IPv4 address; if not it's not valid (like 1.1 or invalid.12).

 

[PaulB]

Which "proposed solution" are you refering to? I didn't propose a solution

I was referring to the PHP filter.

the suggestion made by @Xon shouldn't reject valid email addresses (unless there is a (temporary) failure in DNS resolution or under "obscure" setups were PHP can't resolve but the SMTP server can).

For the most part, but there are always going to be edge cases that seem obvious in hindsight but are difficult to enumerate when actually coding. The most straightforward example is an IP address, which won’t have any DNS records.

Ultimately, you’re still going to end up with invalid email addresses. People make typos, and it’s not always in the hostname portion. Email confirmation is the best way around that, although it has the caveat that an email address that’s valid today might not be valid tomorrow.

As said before, I'd rather prefer such "false rejections" instead of users registering with invalid emails (due to typos).

I disagree and would prefer this to be handled by an add-on as appropriate for each site. This is trivially solved with vanilla XenForo by requiring email confirmation during registration, though that forgoes the ability to require manual approval during registration.

An email address ending with .1 might be valid if the domain part is a valid IPv4 address; if not it's not valid (like 1.1 or invalid.12).

Right; I think we agree there.

 

[bzcomputers]

I came across this issue because on one of our installations XF erorr log was flooded with

Swift_TransportException: Email to 1@1.1 from 1@1.1 failed: Expected response code 354 but got code "554", with message "554 5.5.1 Error: no valid recipients "

Accepting invalid email adresses is problematic because it creates support workload, either for our ops team (in case of such errors being logged) or account managers (in case users accidently mistype an email address).

I'd therefore much rather prefer XenForo to reject some "edge case" emails as invalid (which most likely would happen seldom) than having users contact support after having registered with an invalid email address (which happens frequently).

Been dealing with this for nearly a year now. I have a site with a high number of daily registrations and there are always a few invalid emails everyday that fill the error log with the above "Expected response code 354...". Hoping XenForo's move to Symfony Mailer in 2.2.12, or more likely 2.3, will possibly at least fix the recurring error issue.

 

[Kirby]

Hoping XenForo's move to Symfony Mailer in 2.2.12, or more likely 2.3, will possibly at least fix the recurring error issue.

Symfony Mailer does throw a pretty similar exception:

Expected response code "250/251/252" but got code "501", with message "501 5.1.3 Bad recipient address syntax"

So except for checking code 501 (which could mean everything but the kitchen sink), I don't think there is much XenForo could do at this point (apart from ignoring the error and / or treating it as a bounce).

BTW: Even 1@127.0.0.1 does trigger the same exception whereas 1@[127.0.0.1] does work just fine (as it conforms to section 4.1.3 of RFC 5321?).

 

[Kirby]

The proposed solution rejects valid email addresses, though, which seems even worse.

I've also reported this issue upstream and it seems like EmailValidator 3.0+ (which is a dependency of Symfony Mailer) does reject 1@1.1 as invalid.

So unless XenForo does take extra steps to undo that, future XenForo versions (which use Smyfony Mailer) will reject 1@1.1 as invalid as well

 

[PaulB]

I've also reported this issue upstream and it seems like EmailValidator 3.0+ (which is a dependency of Symfony Mailer) does reject 1@1.1 as invalid.

Interesting. I don't actually see how 1.1 could be valid, but I've learned to stop making assumptions about hostname validity across the board. I wonder what criteria they're using to reject it and whether they also reject valid hostnames. Plus, keep in mind that hostname validity criteria are constantly changing: the last significant change occurred on 2021-10-01, when previously-grandfathered domains were marked invalid by the CA/B Forum. This has affected real-world domain names. Domain validation is hard.

It's technically possible to perform a DNS query for 1.1. While it's unlikely that 1.1 will ever be treated as valid by mainstream software, it's a fairly extreme example, and there are better examples of domains that used to be invalid but are now valid (e.g., domains with emoji in them).

Since there aren't any negative repercussions for allowing users to enter invalid email addresses--they won't be able to confirm their account, after all--I'm still in favor of erring on the side of false negatives when it comes to email validation.

 

[PaulB]

I've also reported this issue upstream and it seems like EmailValidator 3.0+ (which is a dependency of Symfony Mailer) does reject 1@1.1 as invalid.

Where are you encountering this? \XF\Validator\Email::isValid uses \Egulias\EmailValidator\Validation\RFCValidation, which accepts 1@1.1 in my test environment:


Swift seems to exclusively use RFCValidation as well:


I'm not able to see how XF will interface with Symfony Mailer in the future, but, presumably, it will offer similar options and default to the more permissive RFCValidation validator, which will allow 1@1.1. Unless XF's isValid method changes, it will continue accept that email address.

 

[Kirby]

Why is 1@1.1 valid? · Issue #337 · egulias/EmailValidator

The address 1@1.1 is reported as valid, but doesn't seem to be accepted by SMTP servers. Is this a bug?

github.com

I didn't actually test if

v3 is closer to RFC 1035 so it might come out as invalid, for instance.

actually does cause validation to report invalid, I just assumed it would the case

 

[PaulB]

RFC 1035 is very outdated in terms of modern hostname validation. Following it will result in valid domains being marked invalid and vice versa. Domain validation is also context-dependent these days.

For example, _dmarc.xenforo.com is a valid domain that actually exists, and if you receive an email from these forums, your receiving mail server will likely query it. However, RFC 1035 2.3.1 discourages labels beginning with anything other than [A-Za-z]:

<domain> ::= <subdomain> | " "

<subdomain> ::= <label> | <subdomain> "." <label>

<label> ::= <letter> [ [ <ldh-str> ] <let-dig> ]

<ldh-str> ::= <let-dig-hyp> | <let-dig-hyp> <ldh-str>

<let-dig-hyp> ::= <let-dig> | "-"

<let-dig> ::= <letter> | <digit>

<letter> ::= any one of the 52 alphabetic characters A through Z in
upper case and a through z in lower case

<digit> ::= any one of the ten digits 0 through 9

RFC 1035 doesn't actually enforce this syntax: it's in a section titled "Preferred name syntax" (a child of the "Conventions" section), and the text within that section makes it clear that the syntax above is just a general suggestion for end users, not something enforced by DNS. The rest of RFC 1035 fully supports performing queries for domains that violate this suggested syntax, including _dmarc.xenforo.com and 1.1.

Again, RFC 1035 is quite outdated at this point, and while it's an important base for understanding modern DNS, we've come a long way from 1035.

I would be quite surprised if RFCValidation marks 1.1 as invalid, especially if it's attempting to follow RFC 1035.

 

[Kirby]

I would be quite surprised if RFCValidation marks 1.1 as invalid, especially if it's attempting to follow RFC 1035.

Yeah, it doesn't - as said before I assumed taht the author's reply would be correct

Anyway, I still think such email addresses shoudn't be allowed.

Since there aren't any negative repercussions for allowing users to enter invalid email addresses--they won't be able to confirm their account, after all

That is ecavtly the problem why I would prefer such emails to be rejected. If users (accidently) mistype emails when they register an account they might use the contact form (or send an email) - this causes support workload.

But if an email does get rejected telling them that it is invalid they hopefully would be able to realize their mistake and fix it - without requiring support.

 

[PaulB]

Anyway, I still think such email addresses shoudn't be allowed.

I think that should be at the discretion of each site, either via an option in admin.php or an add-on. There are definitely forums that would prefer to allow registration with non-ICANN TLDs. A popular approach for avoiding conflict with ICANN TLDs is to use TLDs that are valid within DNS but invalid according to ICANN's standards: for example, ICANN doesn't allow single-letter TLDs.

That is ecavtly the problem why I would prefer such emails to be rejected. If users (accidently) mistype emails when they register an account they might use the contact form (or send an email) - this causes support workload.

This could be mitigated by sending the confirmation email synchronously and reporting back to the user with an error if the send fails; that's a better solution than imposing incorrect email validation on all XenForo customers. If you want minimal impact and aren't too worried about users being able to trigger a lot of DNS queries, you could trivially swap out RFCValidation for DNSCheckValidation.

 

[Ivancas]

It would be nice if XF checked that the domain part resolves to an A/AAAA/MX record, this would catch a number of invalid domains without needing to code explicit validation

This would be definitely great. Have you started a suggestion thread?

 

[bzcomputers]

This would be definitely great. Have you started a suggestion thread?

Cut down on fake emails, do a DNS check on email domain before allowing registration to be submitted

It would be nice if email addresses submitted in either the registration and/or contact forms were DNS checked to validate that the address is associated with a known good domain excepting email prior to allowing the registration/contact form to even be submitted. This would help cut down on...

xenforo.com

 

[MySiteGuy]

I totally disagree here.

I came across this issue because on one of our installations XF erorr log was flooded with

Swift_TransportException: Email to 1@1.1 from 1@1.1 failed: Expected response code 354 but got code "554", with message "554 5.5.1 Error: no valid recipients "

Accepting invalid email adresses is problematic because it creates support workload, either for our ops team (in case of such errors being logged) or account managers (in case users accidently mistype an email address).

I'd therefore much rather prefer XenForo to reject some "edge case" emails as invalid (which most likely would happen seldom) than having users contact support after having registered with an invalid email address (which happens frequently).

Odd, because it's so rare to see errors from invalid email addresses on my forums that I can't recall when it last happened.