XF 1.5 Email users (massmailing) errors/ .... 4.7.1 Error: too much mail from IP....

[snoopy5]

XF 1.5.17

Hi,

I switched my webhoster 2 weeks ago and today I sent for the first time my weekly newsletter out. I do send it always via SMTP, not by php. See screenshot with my settings in ACP/options/email:



These are the Port settings of the new webhoster:




It took really long to send those emails to around 26.000 users. A lot longer than with the old webhoster. I guess the new one has more limitations like 1000 emails per 10 minutes or so. It took around 3,5 hours.

What does concern me are the error logs in XF-ACP. I have now around 8000 errors. They seems to be alsways duplicates. So in reality "only" 4000 errors for 4000 different email addresses (what a nightmare). The error message is always the same. See below.

Server Error Log
Error Info
Zend_Mail_Protocol_Exception: Email to xyz@gmail.com failed: 4.7.1 Error: too much mail from myIPaddress - library/Zend/Mail/Protocol/Abstract.php:431
Generated By: admin, 22 minutes ago
Stack Trace

#0 /myserverpath/library/Zend/Mail/Protocol/Smtp.php(261): Zend_Mail_Protocol_Abstract->_expect(Array, 300)
#1 /myserverpath/library/Zend/Mail/Transport/Smtp.php(224): Zend_Mail_Protocol_Smtp->mail('bounceforen+6c4...')
#2 /myserverpath/library/Zend/Mail/Transport/Abstract.php(348): Zend_Mail_Transport_Smtp->_sendMail()
#3 /myserverpath/library/Zend/Mail.php(1194): Zend_Mail_Transport_Abstract->send(Object(Zend_Mail))
#4 /myserverpath/library/XenForo/Deferred/UserEmail.php(168): Zend_Mail->send(Object(Zend_Mail_Transport_Smtp))
#5 /myserverpath/library/XenForo/Deferred/UserEmail.php(62): XenForo_Deferred_UserEmail->_sendEmail(Array, Array, Object(Zend_Mail_Transport_Smtp))
#6 /myserverpath/library/XenForo/Model/Deferred.php(295): XenForo_Deferred_UserEmail->execute(Array, Array, 8, '')
#7 /myserverpath/library/XenForo/Model/Deferred.php(429): XenForo_Model_Deferred->runDeferred(Array, 8, '', false)
#8 /myserverpath/library/XenForo/Model/Deferred.php(374): XenForo_Model_Deferred->_runInternal(Array, 8, '', false)
#9 /myserverpath/library/XenForo/ControllerAdmin/Tools.php(159): XenForo_Model_Deferred->run(true, NULL, '', false)
#10 /myserverpath/library/XenForo/FrontController.php(369): XenForo_ControllerAdmin_Tools->actionRunDeferred()
#11 /myserverpath/library/XenForo/FrontController.php(152): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#12 /myserverpath/admin.php(13): XenForo_FrontController->run()
#13 {main}

Request State

array(3) {
  ["url"] => string(55) "https://www.mydomain.com/admin.php?tools/run-deferred"
  ["_GET"] => array(1) {
    ["tools/run-deferred"] => string(0) ""
  }
  ["_POST"] => array(3) {
    ["redirect"] => string(59) "https://www.mydomain.com/admin.php?users/email&sent=25062"
    ["execute"] => string(1) "1"
    ["_xfToken"] => string(8) "********"
  }
}

I then checked the log for bounced email address. But there is not a single bounce recorded. This seems to me very unrealistic.

So my questions are:

1. Why are there no bounce emails? Could the reason for this be a wrong configuration in ACP or a webhoster problem?
2. Why this error above with too many emails from my IP address? Are the tougher restrictions (#emails/10 minutes) from my webhoster a reason for this?

I NEVER had this before with the old webhoster and I send out these newsletter now for several years. I called the new webhoster an d he said he can not do anything, i would just send out too many emails. I can hardly believe that.

 

[snoopy5]

I need to send out today another Newsletter for another Forum with the same webhoster. But before I do this, I need to be sure what can be improved to avoid the same thing

 

[TPerry]

Check with your hosting provider. Looks like they "throttle" your SMTP email sends and you are overloading it.

https://xenforo.com/community/threads/too-much-mail-server-error.113144/

 

[snoopy5]

Check with your hosting provider. Looks like they "throttle" your SMTP email sends and you are overloading it.

https://xenforo.com/community/threads/too-much-mail-server-error.113144/

so this is then not a message from the email receiver's mail provider (the email provider of teh affected users)? Only a message of my own webhoster?

Does that mean, theses emails still got through to those users, only later than expected?

 

[snoopy5]

Where can I check how many emails xenforo tries to send per minue or per batch when using the mass email fetaure within ACP?

Is there a way to change in the code of Xenforo this to a lower amount? (If I only split .i.e by registration date the amount of emails, this does not affect the amount per minute sent...)

 

[snoopy5]

update:

I talked with my webhoster (shared hosting) and he sais that they allow 1000 emails within 10 minutes and everything else will not processed. According to him, if an email is not processed, they send an error message (see above) back to me. This sounds not like throtteling, it sounds like killing everything over 1000 in 10 minutes.

How can I check whether he is right? Sounds all unlogical to me, since I "only" got 4000 error message from the webhoster out of 26.000 emails I sent within 3.5 hours.

My question is :

What is Xenforo doing with emails, it tries to send out but which are blocked then by the webhoster?

An example with numbers: The Xenforo scripts tries to send out around 20 email per second (SMTP)= 12.000 within 10 minutes. The first 1000 get through the webhoster filter. The other 11.000 don't.

What happens now with those 11.000? I do not see anywhere error of 11.000 emails- Does Xenforo then retries to send those 11.000 later on?


Addendum:

If I put the numbers differently and assuming the webhoster is wrong with what he is telling me, it looks like this:

They do allow only 1000 emails/10 minutes slot. But they do NOT block the others. They simply add them to the next 10 minutes slot. This would make more sense. Lets calculate that.

It took me 3.5 hours to send out almost 26.000 emails. Assuming that each 10 minutes, the webhoster let 1000 through, I end up with 21.000 emails within 3.5 hours. So a little bit over 4000 emails got not sent (25.xxx - 21.000 = 4.xxx) and this is the number of errors I got in the ACP error log.

So I need to find n a file where to see those numbers (21.000 sent) - Where can I find it?

And how to send again the same email to those 4000 users which did not get one?

 

[TPerry]

That's one of the downfalls of SMTP. You are at the mercy of whom you connect to for sending restrictions.
If using Postfix you can custom configure it to throttle the amount sent in a time period. That way it will keep them in a queue and send as allowed by it's restrictions.

You need to either have your hosting provider remove the throttling from your account (likely a spam reduction/prevention measure they have in place) or find an SMTP service provider that is more targeted towards mass blast type email programs (like newsletters, etc).

 

[snoopy5]

....
If using Postfix you can custom configure it to throttle the amount sent in a time period. That way it will keep them in a queue and send as allowed by it's restrictions.
.

Never heard of Postfix. Is this a free service?

....
...or find an SMTP service provider that is more targeted towards mass blast type email programs (like newsletters, etc).

Any recommendations?

 

[TPerry]

Never heard of Postfix. Is this a free service?

It's an internal MTA for a server. Use the default process, but it requires it to be configured correctly - usually requiring a VPS or a dedicated server.

Any recommendations?

Sparkpost & SendGrid are two that I'm aware of.

 

[snoopy5]

o.k., those are no options for me.

So back to the original questions:


1. What is Xenforo doing with emails, it tries to send out but which are blocked then by the webhoster?

2. I need to find a logfile on my server with which I can check the numbers the webshoter has told me - Where can I find this?

3. I have now an in the XF ACP an error log of around 4000 not sent emails. How to send again the same email to those 4000 users

4. Where in the code of Xenforo (XF 1.5.17) can I change the numbers so that it is sending always less emails per second or per minute or whatever the slot each time is?

 

[snoopy5]

anybody?