Fixed 'Email Users' uses illegal methods

[Alpha1]

Affected version

2.0 beta 6

I am not sure if this is a technical bug or if there was a conscious decision not to include the legal requirements. If the latter is the case then this is an improvement request:

When sending email through /admin.php?users/email
And selecting 'Include default email wrapper'
The email received does not include the details and functionality required by law as defined by CAN-SPAM, DPEC, DPA, etc. Should this not be in the defulat wrapper?
A handy overview of legal requirements is here: https://litmus.com/blog/the-ultimate-guide-to-international-email-law-infographic

It also seems that no bounce functionality is applied to this process, which means that a board will illegally keep spamming the same email accounts when the email provider sends notifications to stop doing so.

All email sent by XenForo needs to confirm to law and should also avoid the site from being blacklisted as a spammer.

A bare bones email message that doesn't comply with the law nor the requirements of email recipients should really not be in XenForo 2 in 2017.

 

[Chris D]

It also seems that no bounce functionality is applied to this process, which means that a board will illegally keep spamming the same email accounts when the email provider sends notifications to stop doing so.

Just wanted to point out that this isn't accurate. The bounce handling stuff is always applied to all outgoing emails.

 

[Sim]

The email received does not include the details and functionality required by law as defined by CAN-SPAM, DPEC, DPA, etc. Should this not be in the defulat wrapper?

The problem is that you are assuming that all emails sent via that system are necessarily "marketing" emails rather than important system messages or announcements. You don't need to comply with anti-spam laws when someone has signed up to your website and is receiving messages relating to using that service.

Yes, you can also send marketing related emails - which should comply - but not all of them are.

 

[Chris D]

^ That's mostly why I haven't commented on the rest of it.

We'll look into it before commenting in detail, but the bulk of this is ultimately a Suggestion rather than a bug, and it depends very much on how the system is used.

 

[Alpha1]

I'm not assuming marketing per sé but if a board is commercial then its email would indeed be under more stringent regulation. Regardless of mailing purpose.

None the less the email template needs to have all the legal requirements.

As the normal email template has an unsubscribe link and other required information (to/from who, why is it sent) and it is listed that this template is applied I find it strange that the email does not have the same template as other xenforo email.

There is a legal requirement for this. Even for non-commercial email. But its a non-argument. Commercial entities make use of xenforo just as as non-commercial entities do. Else there should be a disclaimer that XF is legally not suited for commercial entities. Which I think would be unwanted.

 

[Sim]

No, there is no legal requirement for a "commercial entity" to add unsubscribe links to transactional or relationship related messages which are a result of a user explicitly signing up to a service.

From: https://www.ftc.gov/tips-advice/business-center/guidance/can-spam-act-compliance-guide-business

Q. How do I know if the CAN-SPAM Act covers email my business is sending?​

A. What matters is the “primary purpose” of the message. To determine the primary purpose, remember that an email can contain three different types of information:​

• Commercial content – which advertises or promotes a commercial product or service, including content on a website operated for a commercial purpose;
• Transactional or relationship content – which facilitates an already agreed-upon transaction or updates a customer about an ongoing transaction; and
• Other content – which is neither commercial nor transactional or relationship.

If the message contains only commercial content, its primary purpose is commercial and it must comply with the requirements of CAN-SPAM. If it contains only transactional or relationship content, its primary purpose is transactional or relationship. In that case, it may not contain false or misleading routing information, but is otherwise exempt from most provisions of the CAN-SPAM Act.​

​

Q. How do I know if what I’m sending is a transactional or relationship message?​

A. The primary purpose of an email is transactional or relationship if it consists only of content that:​

1. facilitates or confirms a commercial transaction that the recipient already has agreed to;
2. gives warranty, recall, safety, or security information about a product or service;
3. gives information about a change in terms or features or account balance information regarding a membership, subscription, account, loan or other ongoing commercial relationship;
4. provides information about an employment relationship or employee benefits; or
5. delivers goods or services as part of a transaction that the recipient already has agreed to.

Marketing or promotional messages do need to comply. Announcements, changes to terms and conditions, security notifications and so on, do not.

 

[Alpha1]

There are two aspects to sending out bulk email:

1. Email Providers & Spam Blacklists
If you want your email to keep arriving to gmail, hotmail/outlook, gmx, icloud, aol, etc and without getting blacklisted then you will have to comply with their requirements. Some examples:
Google:

Unsubscribing
A user must be able to unsubscribe from your mailing list through one of the following means:

• A prominent link in the body of an email leading users to a page confirming his or her unsubscription (no input from the user, other than confirmation, should be required).
• By replying to your email with an unsubscribe request.

https://support.google.com/a/answer/81126?hl=en

GMX:

Opt-out

• The recipient must be given a fast and simple opt-out option for receipt of the mass mail (newsletter, advertising etc.).
• Each email should contain a note to this effect; however, an opt-out option can be provided in the form of a valid reply address.

https://postmaster.gmx.com/en/best-practice
GMX is a popular email provider located in Germany / EU.

AOL:

EASY UNSUBSCRIBE:

• Provide an obvious and visible unsubscribe process in your mail.
• Make it easy for users to unsubscribe from your mailing list.
• Ensure the unsubscribe process is easy to use, such as a one-click unsubscribe web page.
• Users should not have to log into a website in order to unsubscribe.
• Process unsubscribes immediately

SPAM COMPLAINTS:
When users click "report spam", you can get a copy of the spam complaint through our Feedback Loop (FBL) system. Ensure that you have an active FBL on each of your IPs and that you’re processing the complaints quickly. Many senders will treat a spam complaint as an unsubscribe and remove a name from their mailing list if the user clicks "report spam." There are a number of third party tools available to help you manage your FBL.

https://postmaster.aol.com/best-practices

If bulk email methods do not comply with this, then you are likely to find your site on the spam blacklists like barracuda, invaluement, spamcop, spamhaus, SURB, etc. Blacklisting affects all your xenforo email negatively.

2. The Law in all countries of the World.
Current and (up)coming laws that regulate how bulk email can be sent. As email continues to be plagued by SPAM this will only tighten up more and more. A few examples:

USA
While the CAN SPAM act exempts transactional emails, if the bulk email advertises commercial things like account upgrades, content about products or services, product reviews, pages with advertising then opt-out is required.
https://www.ftc.gov/tips-advice/business-center/guidance/can-spam-act-compliance-guide-business

Canada:

No type of organization, including charities and non-profit organizations, is exempt from Canada's Anti-Spam legislation.
If you use electronic channels to promote or market your organization, products or services, Canada's new Anti-Spam Law may affect you.

Promotion here clearly includes non-commercial promotion of your forum.
http://fightspam.gc.ca/eic/site/030.nsf/eng/00286.html

Contents of message
(2) The message must be in a form that conforms to the prescribed requirements and must
(a) set out prescribed information that identifies the person who sent the message and the person — if different — on whose behalf it is sent;
(b) set out information enabling the person to whom the message is sent to readily contact one of the persons referred to in paragraph (a); and
(c) set out an unsubscribe mechanism in accordance with subsection 11(1).

http://laws-lois.justice.gc.ca/eng/acts/E-1.6/page-2.html#h-6

EU

The data subject shall have the right to withdraw his or her consent at any time. The withdrawal of consent shall not affect the lawfulness of processing based on consent before its withdrawal. Prior to giving consent, the data subject shall be informed thereof. It shall be as easy to withdraw as to give consent.

http://ec.europa.eu/justice/data-protection/reform/files/regulation_oj_en.pdf
This also relates to the use of data such as email addresses within the scope electronic communication. We are data controllers as such. Next year the grace period of the GDPR end and then we should see enforcement of it. Then it will also become clear what the interpretations of regulators are. One thing is clear, this regulation stretches worldwide to anyone with subscribers in the EU. More from the same regulation:

When developing, designing, selecting and using applications, services and products that are based on the processing of personal data or process personal data to fulfil their task, producers of the products, services and applications should be encouraged to take into account the right to data protection when developing and designing such products, services and applications and, with due regard to the state of the art, to make sure that controllers and processors are able to fulfil their data protection obligations.

A worldwide review of all countries would be nice and would further clarify it. I have done it years ago and its a monks task. I hope that a few examples demonstrates the need.

The functionality already exists for all other xenforo email functions. It woudl be good practise and it would make sense to have it here as well.
I am not using the function to email my 190k subscribers, because I am not taking the risk.

 

[DragonByte Tech]

There are two aspects to sending out bulk email

I think you missed the point of @Sim's post. You appear to be claiming that 100% of XenForo's emails are subject to the laws you have listed, which is demonstrably false. The only emails being sent from XenForo that are subject to the anti-spam laws you have quoted are the "Email Users" emails, and ONLY if they are being used to send emails that do not pertain to the account the users signed up for.

In other words, consider the following emails:

1. "Check out what's new on $forumName!"
2. "Your password reset information"
3. "$forumName weekly mailing list"
4. "$forumName account activation instructions"

Can you guess which of these emails are subject to spam laws? If you guessed anything other than #1 and #3, congratulations, you're wrong

In fact, the very text you quote proves Sim's point; BULK email. Sending password reset emails, sending account activation instructions, sending receipts for account upgrade purchases, [...] - these are NOT BULK EMAIL.

Do I agree with you that email sent via XF's "send email to all users" should have easy-unsubscribe functionality (e.g. a unsubscription hash similar to our DragonByte Mail product, that will unsubscribe users without requiring a login)? Of course I do, as does every sane administrator.

Do I agree with you that there should be unsubscription links for account emails (transactional emails)? Hell no. Being able to say "I do not wish to receive emails regarding my password reset" is the most idiotic thing since metal chewing gum.

I have absolutely no idea why you decided to spend that much time finding all those law links, when literally no-one disagrees with you that XF-generated bulk email should have unsubscribe functionality. We are disagreeing with you in saying that all XF-generated email is bulk email (it isn't).


Fillip

 

[Alpha1]

You appear to be claiming that 100% of XenForo's emails are subject to the laws you have listed, which is demonstrably false.

I can see how my my words are taken that way and I should have worded it clearer. But no, this thread is not about 100% of xenforo email. It pertains to the specific functionality of sending email through admin.php?users/email

This functionality can be used in multiple ways but most importantly to send bulk mail to the subscribed users. i.e. whats new on the site, promotional content, benefits of account upgrades and similar. This has nothing to do with password resets, receipts for account upgrade purchases.

We are disagreeing with you in saying that all XF-generated email is bulk email (it isn't).

Of course it isn't. There is no disagreement there.

My point is that if we use the Email Users function to send bulk email to our members, this can and will likely result in problems like spam blacklisting and law breaches. For example if I send a weekly newsletter to 190k subscribers, without the normal template being applied, then within a few months I am likely to see my site blacklisted by email providers or worse. And yes, there is legal risk here as soon as you mail out to one of the countries, or group of countries were the content falls under anti-spam laws or privacy laws. This is why I highlighted some examples where sending bulk mail to subscribed users requires an email template with the various information and unsubscribe link.

If I tick the function 'Include default email wrapper' which is explained as: 'If selected, your email content will be wrapped in the standard header and footer used in emails sent elsewhere in XenForo.' then I expect that functionality being applied. i.e. unsubscribe link, and other specifics to comply with the relevant standards and laws.

But in my tests this doesn't work, which is why I filed this as a bug report. This means that as a result a webmaster can face unexpected consequences of this issue which negatively affect the site by blacklisting or law breach.

Mind that not all webmasters have equal experience and many will simply use the functionality as its presented. Therefore IMHO there should not be such risks involved with using stock functionality.

 

[DragonByte Tech]

I can see how my my words are taken that way and I should have worded it clearer. But no, this thread is not about 100% of xenforo email. It pertains to the specific functionality of sending email through admin.php?users/email

Fair enough, I admit I must have skimmed over your post because I do see it there, so mea culpa on that one.

If I tick the function 'Include default email wrapper' which is explained as: 'If selected, your email content will be wrapped in the standard header and footer used in emails sent elsewhere in XenForo.' then I expect that functionality being applied. i.e. unsubscribe link, and other specifics to comply with the relevant standards and laws.

The reason why this is a suggestion and not a bug report in the eyes of the developers, is the fact that that's an unreasonable expectation.

To clarify; it's not unreasonable to expect these features to exist, but it is unreasonable to assume that "add the standard header and footer" means "add functionality that is not found in the standard header and footer".

"Standard header and footer" means exactly that, add the standard stuff that's applied to all other XF email (which, as we both agree on, is transactional and thus exempt). The option is designed to allow admins to create their own HTML newsletters if they want to, instead of having the XF CSS applied.

If the standard header and footer is only ever applied to transactional email in every other use case, I would be very interested in hearing why exactly you believed that the wording meant something else.

I can't speak for the XF developers, but I'd wager they designed the "email users" functionality with the notion in mind that it is not a full-featured newsletter solution and that the admins take responsibility for any problems that arise from misuse of that functionality, in the same way that the developers of an SSH connection application does not assume responsibility if an admin accidentally rm -rf / 's their own server.

Speaking for myself, if someone using our products notices that it doesn't do what they thought it would do, I don't appreciate it if they report the missing functionality as a bug. Nine times out of ten, it is not a bug, and labelling it as such is to me the same as saying "obviously it should work like this, surely you couldn't be that incompetent that you didn't add the functionality... right?"

Give your developer pals the benefit of the doubt and maybe think they just didn't design it with X or Y in mind, so post a feature request or a suggestion instead of a bug


Fillip

 

[Alpha1]

"Standard header and footer" means exactly that, add the standard stuff that's applied to all other XF email

This is incorrect. XenForo Emails for subscribed content have this footer:

Please do not reply to this message. You must visit the forum to reply.

This message was sent to you because you opted to watch the forum "XenForo questions and support" at XenForo community with email notification of new threads or messages. You will not receive any further emails about this thread until you have read the new messages.

If you no longer wish to receive these emails, you may disable emails from this forum or disable all emails.

The bulk email in 'Email Users' is aimed at subscribed users. Following the same logic I was expecting a similar footer to be added to the bulk email. After all XF already has an unsubscribe link function for the bulk email feature. This is why I expected it to be added by 'Include default email wrapper. (If selected, your email content will be wrapped in the standard header and footer used in emails sent elsewhere in XenForo)'

I had not considered that the term 'standard footer' would relate to a footer that is just a link.

surely you couldn't be that incompetent that you didn't add the functionality... right?"

I dont think that the XF developers are oversensitive. None of my suggestions or bug reports are intended that way. XF is great platform and I have a deep respect for their work. Not every part of XF can be fully matured. It seems to me that the bulk mail feature just has not gotten much attention, because focus was on other cool features they have added.

 

[Mike]

There are some improvements that we can make here and should, so I will leave this open. I'm not entirely sure the specific format that they'll take yet.

However, I do feel that the assumption that "Include default email wrapper" would inherently add a whole section regarding unsubscribe to be unexpected in my eyes, especially as we can't necessarily know the content of the message and whether an unsubscribe link is appropriate. The example unsubscribe content in watched emails is really part of the content itself rather than a "global" footer.

As an additional note, you can generate an unsubscribe link by adding {unsub} into the email (it's one of the listed tokens). This is something that wasn't really an option in XF1.

 

[ŽivaAkcija]

i dont know it proper to post it here but can you consider to add option for disposable/temp mail in proces of registration to check on it and not allow any registration with this email?

 

[Chris D]

You'd just use the banned email function for that and you'd ban all known disposable email domains.

XF2 includes the facility to export and import XML files containing lists of banned emails, IPs etc. so it makes it easier for a list to be maintained and easily shared and imported by others.

Aside from that advice, it's a totally different thing to what we're discussing here so you'd need to post a Suggestion (though one might exist already).

 

[Alpha1]

Thanks @Mike !

It doesn't really matter what form the improvements will have. its the result that counts. It would be optimal if there would be a way to include the aforementioned requirements like sender details and unsubscribe link. For example by adding an additional checkbox.

 

[Mike]

Realistically, I think there is really only one new thing that was really needed here. There is now a checked-by-default "Automatically include an unsubscribe link" when on the email users setup page. If you uncheck it or use {unsub} in the body, it will be ignored. If it's applicable, it will include an "unsubscribe from mailing list" link at the body (or the main "body" section of the email).

Other than that, most of the necessarily functionality was there. As noted, the bounce handling on these emails is no different than any other XF email.

We have already had two phrases that you can customize to add extra details to the email footer (email_footer_html, email_footer_text); if you customize these, they'll be included in every email sent by XF (as part of the wrapper). You can include an address here if needed, for example. We have how made it easier to find this. When editing a language, we now pick out the current value for a few commonly edited phrases (the above mentioned ones, terms_rules_text, extra_copyright) and display them along with the other language settings. (Note that we don't do this when creating a language because their values depend heavily on the language's parent.)

 

[Alpha1]

Thank you @Mike
Consider to add this information to the manual, so that the information is easily available. I didnt know about the existence of email_footer_html, email_footer_text.