Partial fix Job XF:EmailUnsubscribe: Invalid header value detected

[Nirjonadda]

Affected version

2.1.7

Server error log:

Zend\Mail\Header\Exception\InvalidArgumentException: Job XF:EmailUnsubscribe: Invalid header value detected src/vendor/zendframework/zend-mail/src/Header/GenericHeader.php:63

Generated by: Unknown account Mar 9, 2020 at 9:43 AM

Stack trace

#0 src/vendor-patch/Zend/Mail/Headers.php(481): Zend\Mail\Header\GenericHeader::splitHeaderLine('X-Ham-Report: S...')
#1 src/vendor-patch/Zend/Mail/Headers.php(232): Zend\Mail\Headers->loadHeader('X-Ham-Report: S...')
#2 src/vendor-patch/Zend/Mail/Headers.php(96): Zend\Mail\Headers->addHeaderLine('X-Ham-Report: S...')
#3 src/vendor/zendframework/zend-mime/src/Decode.php(149): Zend\Mail\Headers::fromString('Return-Path: <t...', '
')
#4 src/vendor/zendframework/zend-mail/src/Storage/Part.php(110): Zend\Mime\Decode::splitMessage('Return-Path: <t...', 'Return-Path: <t...', '')
#5 src/vendor/zendframework/zend-mail/src/Storage/Message.php(52): Zend\Mail\Storage\Part->__construct(Array)
#6 src/vendor/zendframework/zend-mail/src/Storage/Imap.php(126): Zend\Mail\Storage\Message->__construct(Array)
#7 src/XF/EmailUnsubscribe/Processor.php(38): Zend\Mail\Storage\Imap->getMessage(1)
#8 src/XF/Job/EmailUnsubscribe.php(27): XF\EmailUnsubscribe\Processor->processFromStorage(Object(Zend\Mail\Storage\Imap), G)
#9 src/XF/Job/Manager.php(253): XF\Job\EmailUnsubscribe->run(G)
#10 src/XF/Job/Manager.php(195): XF\Job\Manager->runJobInternal(Array, G)
#11 src/XF/Job/Manager.php(79): XF\Job\Manager->runJobEntry(Array, G)
#12 job.php(42): XF\Job\Manager->runQueue(false, 8)
#13 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(23) "/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}

 

[Nirjonadda]

One more Server error log:

Zend\Mail\Exception\RuntimeException: Bounce message processing failed: Line "550-5.1.1 double-checking the recipient's email address for typos or" does not match header format! src/vendor-patch/Zend/Mail/Headers.php:110

Generated by: Unknown account Mar 10, 2020 at 12:23 AM

Stack trace

#0 src/vendor/zendframework/zend-mime/src/Decode.php(149): Zend\Mail\Headers::fromString('Action: failed
...', '
')
#1 src/XF/EmailBounce/Parser.php(385): Zend\Mime\Decode::splitMessage('Action: failed
...', 'Action: failed
...', NULL)
#2 src/XF/EmailBounce/Parser.php(208): XF\EmailBounce\Parser->parseDeliveryStatus(Object(Zend\Mail\Storage\Message), Object(XF\EmailBounce\ParseResult))
#3 src/XF/EmailBounce/Processor.php(92): XF\EmailBounce\Parser->parseMessage(Object(Zend\Mail\Storage\Message))
#4 src/XF/EmailBounce/Processor.php(68): XF\EmailBounce\Processor->processMessage('Return-Path: <0...')
#5 src/XF/Job/EmailBounce.php(27): XF\EmailBounce\Processor->processFromStorage(Object(Zend\Mail\Storage\Imap), G)
#6 src/XF/Job/Manager.php(253): XF\Job\EmailBounce->run(G)
#7 src/XF/Job/Manager.php(195): XF\Job\Manager->runJobInternal(Array, G)
#8 src/XF/Job/Manager.php(79): XF\Job\Manager->runJobEntry(Array, G)
#9 job.php(42): XF\Job\Manager->runQueue(false, 8)
#10 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(36) "/threads/1010/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}

 

[Wutime]

I have hundreds of these:

 

[Nirjonadda]

Yes I am also getting lot of Server error log about Job XF:EmailUnsubscribe: Invalid header value detected. I am also reported to cPanel but the said this Xenforo Bug with Zend.

 

[Wutime]

It's definitely something to do with XF. I'm not sure what though.

I'm guessing it's something to do when XF sends out emails for thread notices or similar.

 

[Chris D]

While we're not going to close this bug report yet, I do have to take exception to the idea that an external system (like Spam Assassin) adding clearly invalid X-Ham-Report headers to existing emails is a problem of our making.

To the best of my knowledge, Zend parses the email headers exactly to the spec and the X-Ham-Report header is, in fact, not valid according to that spec, hence the problem.

These relate to emails being sent to the "Unsubscribe email address" you have defined in your settings, but the same can happen with your "Bounce email address" too.

Our recommendation to workaround this for now is to simply disable Spam Assassin for those mailboxes (if possible). XF itself manages the contents of those mailboxes and any emails going to them will be discarded if they aren't relevant to bounces or unsubscribes.

However, ultimately, while the Zend header detection is slightly on the strict side, the root cause of this is the invalid header to start with and it's disappointing that isn't acknowledged.

 

[Wutime]

To the best of my knowledge, Zend parses the email headers exactly to the spec and the X-Ham-Report header is, in fact, not valid according to that spec, hence the problem.

These relate to emails being sent to the "Unsubscribe email address" you have defined in your settings, but the same can happen with your "Bounce email address" too.

Our recommendation to workaround this for now is to simply disable Spam Assassin for those mailboxes (if possible). XF itself manages the contents of those mailboxes and any emails going to them will be discarded if they aren't relevant to bounces or unsubscribes.

I'll give that a go and reply back. Thanks for the heads up.

 

[Nirjonadda]

Our recommendation to workaround this for now is to simply disable Spam Assassin for those mailboxes (if possible). XF itself manages the contents of those mailboxes and any emails going to them will be discarded if they aren't relevant to bounces or unsubscribes.

Spam Assassin already disabled but still generated this server error log. Like one log generated 30 minute, 24 hours > 12 log generated. We have only server error log from unsubscribe email handler, not bounced email handler.

 

[Wutime]

This only started for me since the recent update - never had an issue previously and as far as I know, nothing with our server has changed (though WHM/CPANEL did recently update to v86.0.16 from v86.0.14).

 

[Wutime]

* NOTE - THIS DIDN'T WORK FOR ME, THOUGH SHOULD - IF YOU KNOW HOW TO MAKE THIS WORK PLEASE POST *

* LATER POST HAS A HACK WORK AROUND *

For anyone having this issue I followed the following instructions to remove the: X-Ham-Report from message header under WHM/CPanel

The following documents can help you do this yourself:

1. https://forums.cpanel.net/threads/how-to-remove-x-ham-report-from-message-header.636153/
2. https://documentation.cpanel.net/display/CKB/How+to+Customize+the+Exim+System+Filter+File

To complete this task you do need root access to your server by SSH to create the custom filter, and access to WHM to enable it.

Here are the steps taken:

1. Login as root (or su root)
2. Create a file in /usr/local/cpanel/etc/exim/sysfilter/options
   1. I named this file: remove_x_ham_report_header
3. Edit (nano) this file to include the code shown below
4. Save the file
5. Login to WHM
6. Under "Exim Configuration Manager" -> "Basic Editor" -> "Filters"
7. Near the bottom you need to enable the newly created filter (it'll be disabled by default), it should be named "Custom Filter: remove_x_ham_report_header"

remove_x_ham_report_header code:

#HamRemoval
if
$header_to: is "email@email-domain-test.tld"
then
headers remove "X-Ham-Report"
deliver "email@email-domain-test.tld"
endif

 

[Nirjonadda]

@Wutime Already disabled Apache SpamAssassin. So disabling Apache SpamAssassin are not remove X-Ham-Report from message header?



Does this tutorial stopped generated this server error log? OK I am going to use your in tutorial but what we need replace with test1@email-domain-test.tld and test2@email-domain-test.tld email address? Does this need replace with Unsubscribe email address?

Example:

#HamRemoval
if
$header_to: is "Unsubscribe email address"
then
headers remove "X-Ham-Report"
deliver "Unsubscribe email address"
endif

Also how to use multiple email address for remove the: X-Ham-Report from message header?

 

[Wutime]

Already disabled Apache SpamAssassin. So disabling Apache SpamAssassin are not remove X-Ham-Report from message header?

I'm not exactly sure what the process flow is, but I'm going to test this for 24 hours to see if the error messages subside. Like I'm guessing you're experiencing the errors even after you disabled SpamAssassin? Or was your SA always disabled?

 

[Nirjonadda]

I'm not exactly sure what the process flow is, but I'm going to test this for 24 hours to see if the error messages subside. Like I'm guessing you're experiencing the errors even after you disabled SpamAssassin? Or was your SA always disabled?

Yes I am get generated this server error log after disabled SpamAssassin.

I am going to use your in tutorial but what we need replace with test1@email-domain-test.tld and test2@email-domain-test.tld email address? Does this need replace with Unsubscribe email address?

Example:

#HamRemoval
if
$header_to: is "Unsubscribe email address"
then
headers remove "X-Ham-Report"
deliver "Unsubscribe email address"
endif

Also how to use multiple email address for remove the: X-Ham-Report from message header?

I am also like to test this process but let me confirm what email address you are used? Does this need replace with Unsubscribe email address, Default email address, Contact email address or Bounced email address?

 

[Wutime]

I've tried many things, I'm not able to get the "X-Ham-Report" removed when email arrives from domains outside of my server. Similar to you, it's whether Spam Assassin is enabled or not, and irregardless of trying to have the headers removed once they arrive.

 

[Wutime]

Okay, the following I got to work but this is not an ideal solution. It involves editing the exim.conf directly, which, on system update, maybe overwritten. It did prove effective for removing the X-Ham-Report headers from emails however.

1. Root login to your WHM server
2. Edit /etc/exim.conf
3. Find and comment out the following line shown below
4. Restart exim after you save the file

add_header = X-Ham-Report: $spam_report

Change to:

# add_header = X-Ham-Report: $spam_report

 

[Wutime]

While perhaps not specification, given the number of people running WHM, it would be nice if Xenforo could add/allow "X-Ham-Report" given it's a likely a somewhat common illegal mail header.

 

[m1ne]

It involves editing the exim.conf directly, which, on system update, maybe overwritten.

Use /etc/exim.conf.local to avoid this problem.

 

[Jeremy P]

While perhaps not specification, given the number of people running WHM, it would be nice if Xenforo could add/allow "X-Ham-Report" given it's a likely a somewhat common illegal mail header.

To be clear (and maybe pedantic), the issue isn't that X-Ham-Report is an invalid header name (it's not, non-standard X- headers are not only extremely common but permitted), but that the header has an invalid value. This is something that could (should) be fixed in SpamAssassin.

 

[Nirjonadda]

Okay, the following I got to work but this is not an ideal solution. It involves editing the exim.conf directly, which, on system update, maybe overwritten. It did prove effective for removing the X-Ham-Report headers from emails however.

1. Root login to your WHM server
2. Edit /etc/exim.conf
3. Find and comment out the following line shown below
4. Restart exim after you save the file

add_header = X-Ham-Report: $spam_report

Change to:

# add_header = X-Ham-Report: $spam_report

Such changes are temporary and will be reverted when UPCP or EximUP runs. In order to make the changes permanent, do the following through WHM.

WHM->exim configuration editor and go to advanced editor.

Copy the entire content in default_spam_scan and uncheck it.

Paste the contents into custom_begin_spam_scan but remove the X-HAM line

  warn
     # Remove spam headers from outside sources
     condition = ${perl{spamd_is_available}}
     !hosts = +skipsmtpcheck_hosts
     remove_header  = x-spam-subject : x-spam-status : x-spam-score : x-spam-bar : x-spam-report : x-spam-flag : x-ham-report


  warn
    condition = ${perl{spamd_is_available}}
    condition = ${if eq {${acl_m0}}{1}{1}{0}}
    spam =  ${acl_m1}/defer_ok
    # Always make sure cPanel support mail can get through
    !hosts = : +trustedmailhosts : +cpanel_mail_netblocks
    log_message = "SpamAssassin as ${acl_m1} detected message as spam ($spam_score)"
    add_header = X-Spam-Subject: ***SPAM*** $rh_subject
    add_header = X-Spam-Status: Yes, score=$spam_score
    add_header = X-Spam-Score: $spam_score_int
    add_header = X-Spam-Bar: $spam_bar
    add_header = X-Spam-Report: ${sg{$spam_report}{\N\n \n\N}{\n}}
    add_header = X-Spam-Flag: YES
    set acl_m2 = 1

  warn
      condition = ${perl{spamd_is_available}}
      condition =  ${if eq {$spam_score_int}{}{0}{${if <= {${spam_score_int}}{8000}{${if >= {${spam_score_int}}{50}{${perl{store_spam}{$sender_host_address}{$spam_score}}}{0}}}{0}}}}

  warn
  condition = ${perl{spamd_is_available}}
  condition = ${if eq {${acl_m0}}{1}{${if eq {${acl_m2}}{1}{0}{1}}}{0}}
  add_header = X-Spam-Status: No, score=$spam_score
  add_header = X-Spam-Score: $spam_score_int
  add_header = X-Spam-Bar: $spam_bar
  #add_header = X-Ham-Report: ${sg{$spam_report}{\N\n \n\N}{\n}}
  add_header = X-Spam-Flag: NO
  log_message = "SpamAssassin as ${acl_m1} detected message as NOT spam ($spam_score)"

If you get a config error make sure the line is all on one line. Save it and you are done.

condition = ${if eq {$spam_score_int}{}{0}{${if <= {${spam_score_int}}{8000}{${if >= {${spam_score_int}}{50}{${perl{store_spam}{$sender_host_address}{$spam_score}}}{0}}}{0}}}}

 

[Nirjonadda]

Use /etc/exim.conf.local to avoid this problem.

How can use this to remove X-Ham-Report header? Please provide step by step instructions on how to properly work in /etc/exim.conf.local for remove X-Ham-Report header.