jeffwidman
Active member
Yes. They are still members, and still in your DB, just Xenforo doesn't include users with bounced emails or permabanned users in the publicly listed user count.
Resource icon
[bd] Mails 1.1.0
- Thread starter xfrocks
- Start date
Robby
Well-known member
Ok thanks for the reply.
I didn't know that. Makes sense.
nrep
Well-known member
I've had some users with soft-bounce e-mail problems in Mandrill (all AOL) and they've been blocked from the forum. Is it possible to implement some soft bounce checking system (i.e. max of x soft bounces over y days) the Mandrill integration? I'm happy to pay for custom modifications to the plugin.
clackey
Active member
Ya, I'm having the same issue. I might have to find another addon/service as we've got dozens of aol users stuck in this state each day. I too would be happy to pay for a version that resolves this issue.
jauburn
Well-known member
clove28
Active member
JordanH
Well-known member
This does not seem to be sending out all the emails. I get no errors in ACP.
I just been keeping track on my Mandrill dashboard, and it is sending maybe 1 out of every 10 emails.
Example, someone replies to a thread. Instead of notifying everyone in the thread who is watching, it only notifies one.
I just been keeping track on my Mandrill dashboard, and it is sending maybe 1 out of every 10 emails.
Example, someone replies to a thread. Instead of notifying everyone in the thread who is watching, it only notifies one.
Zynektic
Well-known member
Do you have your Email Options to stop sending emails after x days to non active users? (I think it was those options)
nrep
Well-known member
I'm having an unusual error where bounced e-mails from Mandrill are not marked as invalid. Here's one example:
That's a hard bounce, but when the cron runs it isn't picked up and the user marked as invalid. Is there a way I can troubleshoot this for a fix? I can see that the e-mail this applies to is on the Mandrill Rejection list in the admin console.
That's a hard bounce, but when the cron runs it isn't picked up and the user marked as invalid. Is there a way I can troubleshoot this for a fix? I can see that the e-mail this applies to is on the Mandrill Rejection list in the admin console.
Last edited:
nrep
Well-known member
OK, I've found out what the problem is and it's not related to [BD] Mails - but it is a problem that may affect many other Mandrill users. It turns out that the API call for the rejection list will return a maximum of 1000 results. [BD] Mails calls for all active blacklist results, but these are post-filtered from the first 1000 on the total blacklist (expired/active). That means that if you have over 1000 e-mail addresses that have bounced at any point in time, you won't get the full set of bounced e-mails returned via the API. Here's what Mandrill support said:
The only way around this is to use the CSV, or put in place a workaround they suggested:
Is this problem affecting anyone else?
edit:
I've written a workaround which loops through the first letters of the e-mail, as Mandrill suggested. I'm a novice coder, so not sure if this the best way to do it. Hopefully it will help others with the same problem. Here's the replacement code for /Transport/Mandrill.php:
The only way around this is to use the CSV, or put in place a workaround they suggested:
Is this problem affecting anyone else?
edit:
I've written a workaround which loops through the first letters of the e-mail, as Mandrill suggested. I'm a novice coder, so not sure if this the best way to do it. Hopefully it will help others with the same problem. Here's the replacement code for /Transport/Mandrill.php:
PHP:
public function bdMails_bounceList()
{
if (date("j") % 2 == 0) {
$letters = range('a', 'm');
} else {
$letters = range('n', 'z');
}
// Space out requests to stay under API rate limit
$list = array();
foreach ($letters as $letter) {
$client = XenForo_Helper_Http::getClient(sprintf('%s/rejects/list.json', self::$apiUrl));
$client->setRawData(json_encode(array('key' => $this->_apiKey, 'email' => $letter)));
$response = $client->request('POST')->getBody();
$responseArray = @json_decode($response, true);
if (!empty($responseArray))
{
foreach ($responseArray as $reject)
{
if (is_array($reject))
{
if ($reject['reason'] !== 'hard-bounce')
{
// TODO: should we only get hard bounces?
// continue;
}
$list[$reject['email']] = array(
'email' => $reject['email'],
'reason' => $reject['detail'] ? $reject['detail'] : $reject['reason'],
);
}
}
}
}
return $list;
}
Last edited:
ichpen
Well-known member
Curious if you got this figured out or if anyone on 148 is having issues
Robby
Well-known member
Fir the moment less issues...
BobbyWibowo
Well-known member
I've this installed on 1.4.8 and it works just fine.
EDIT: Can we have bounce handlers for Mailgun?
EDIT: Can we have bounce handlers for Mailgun?
jeffwidman
Active member
I just spent some time debugging issues I was hitting with BDMails / Mandrill.
First, anytime an email address returns a soft-bounce, a hard-bounce, or if the recipient marks the message as spam, then Mandrill adds the user to the Mandrill blacklist. If you attempt to send an email to an address on the blacklist, Mandrill will simply reject it until the address blacklist expires. They're trying to prevent you from shooting yourself in the foot because repeatedly sending email to non-valid addresses will cause receiving mail servers to mark you as a spammer. Spam users have an expiration time of 1 year, while soft/hard bounces have much shorter expiration times that gradually increase as the address continues to bounce.
Somewhat confusingly, once an address has been added to the blacklist, it stays on the blacklist forever. It is not removed when it expires, it only gets its status changed to expired and Mandrill will now allow sending email to that address. It only leaves the blacklist if it is manually removed through the API or web interface.
Here are the three issues that need to be dealt with:
Ultimately, this is an AOL problem, not a Mandrill problem. You'll hit the Mandrill blocklist, but even if you remove the AOL email from Mandrill's blacklist, AOL will still bounce the message. Ask the user to add your sending email address to their contacts, and then remove their address from Mandrill blacklist and mark the account as valid in Xenforo. It's worked for my sites.
First, anytime an email address returns a soft-bounce, a hard-bounce, or if the recipient marks the message as spam, then Mandrill adds the user to the Mandrill blacklist. If you attempt to send an email to an address on the blacklist, Mandrill will simply reject it until the address blacklist expires. They're trying to prevent you from shooting yourself in the foot because repeatedly sending email to non-valid addresses will cause receiving mail servers to mark you as a spammer. Spam users have an expiration time of 1 year, while soft/hard bounces have much shorter expiration times that gradually increase as the address continues to bounce.
Somewhat confusingly, once an address has been added to the blacklist, it stays on the blacklist forever. It is not removed when it expires, it only gets its status changed to expired and Mandrill will now allow sending email to that address. It only leaves the blacklist if it is manually removed through the API or web interface.
Here are the three issues that need to be dealt with:
A) When a recipient marks the message as spam, and Mandrill reports the message as undeliverable due to a spam complaint, BD Mails just gives an error. It doesn't mark the address as invalid/bouncing. While life would be simpler if BDMails marked this as bouncing, technically this is correct, as the address isn't invalid.
On my sites, the emails that were getting marked as 'spam' were all mass-mailings, such as the 'Inactive Users' addon or 'Weekly Digest' addon. The users didn't know how to unsubscribe from these, so just reported them as spam. Ideally, these users still get transactional emails such as notifications of PMs, but don't get newsletters. @AndyB helpfully made it so the 'Weekly Digest' and 'Inactive User' addons respect the 'Receive site mailings' setting.
I handled this the following way:
Probably the best way to handle this is if BD Mails would automatically delete an email from Mandrill's blacklist once it marks it as bouncing in Xenforo. Deleting emails from the blacklist is a small hit to your Mandrill sending reputation, but it's a far better solution than never marking an address as bouncing... because then you just end up re-sending to that email address once it expires on Mandrill's blacklist and get a bigger hit to your reputation. @xfrocks can you fix? Or @Xon (if he uses this) might be able to provide a patch.
You could iterate through all the letters of the alphabet like @nrep 's patch above, but that's buggy because I have emails starting with non-alphabet characters on my blacklist. Simpler to just remove the emails from Mandrill rather than trying to exhaustively iterate through all permutations.
In the meantime, a hacky alternative is to simply clear your Mandrill blacklist whenever it gets larger than 1000. Rather than tediously clicking 'remove' thousands of times, I wrote this quick python script:
C) Finally, I hit multiple bugs with Mandrill:
On my sites, the emails that were getting marked as 'spam' were all mass-mailings, such as the 'Inactive Users' addon or 'Weekly Digest' addon. The users didn't know how to unsubscribe from these, so just reported them as spam. Ideally, these users still get transactional emails such as notifications of PMs, but don't get newsletters. @AndyB helpfully made it so the 'Weekly Digest' and 'Inactive User' addons respect the 'Receive site mailings' setting.
I handled this the following way:
1) exporting my blacklist as a CSV
2) filtering by all users who were reported as 'spam'
3) in Xenforo, go to ACP > User Profile with that email address > Privacy tab > Uncheck 'receive site mailings'
If you have any other mass email addons that don't respect 'receive site mailings', you'll need to prevent them from sending to these addresses.
4) In Mandrill, remove these users from the blacklist, so that Mandrill will still attempt to email them.
B) As @nrep helpfully tracked down, Mandrill only reports the first 1,000 blacklist emails through the API. And despite what their docs imply, the 1,000 emails includes emails that have expired but are still on the blacklist (see explanation at beginning of this post). BD Mails uses this list for seeing who to mark as bouncing, so once you have more than 1,000 bouncing emails, as more emails start bouncing, BD Mails won't realize it. Caution--I think this is what's happening. I hit multiple Mandrill bugs (mentioned below), so I'm not 100% sure this is the problem, but it certainly makes sense from what Mandrill support said and BDMails source code.2) filtering by all users who were reported as 'spam'
3) in Xenforo, go to ACP > User Profile with that email address > Privacy tab > Uncheck 'receive site mailings'
If you have any other mass email addons that don't respect 'receive site mailings', you'll need to prevent them from sending to these addresses.
4) In Mandrill, remove these users from the blacklist, so that Mandrill will still attempt to email them.
Probably the best way to handle this is if BD Mails would automatically delete an email from Mandrill's blacklist once it marks it as bouncing in Xenforo. Deleting emails from the blacklist is a small hit to your Mandrill sending reputation, but it's a far better solution than never marking an address as bouncing... because then you just end up re-sending to that email address once it expires on Mandrill's blacklist and get a bigger hit to your reputation. @xfrocks can you fix? Or @Xon (if he uses this) might be able to provide a patch.
You could iterate through all the letters of the alphabet like @nrep 's patch above, but that's buggy because I have emails starting with non-alphabet characters on my blacklist. Simpler to just remove the emails from Mandrill rather than trying to exhaustively iterate through all permutations.
In the meantime, a hacky alternative is to simply clear your Mandrill blacklist whenever it gets larger than 1000. Rather than tediously clicking 'remove' thousands of times, I wrote this quick python script:
Code:
import mandrill
api_key = 'Your API Key'
m_client = mandrill.Mandrill(api_key)
# print(m_client.users.ping()) # Should return “PONG!”
blacklist_contents = m_client.rejects.list(include_expired=True)
for i in blacklist_contents:
print(m_client.rejects.delete(email=i['email']))C) Finally, I hit multiple bugs with Mandrill:
1) Their web GUI is telling me that there's no one on the blacklist, but if I do a query for all email addresses on the blacklist that start with 'a' using the url Log in to Mandrill I get results, so clearly there's a bug. I can click 'remove' on those results, and the addresses appear to be removed, as they no longer show up on the blacklist.
2) The url Log in to Mandrill gives me an incomplete list of results and doesn't include all addresses starting with 'a' that are listed in the blacklist CSV export.
3) I ran the python script to delete everyone from the blacklist. The API kept telling me it'd deleted all the email addresses, but when I re-exported a CSV of the blacklist, those email addresses weren't removed. Similarly, I could re-run my script, and the API would once again report it'd successfully deleted the email addresses, so clearly they were hanging around.
I've filed bug reports on both issues, and hopefully once they're resolved this can be fixed for good.2) The url Log in to Mandrill gives me an incomplete list of results and doesn't include all addresses starting with 'a' that are listed in the blacklist CSV export.
3) I ran the python script to delete everyone from the blacklist. The API kept telling me it'd deleted all the email addresses, but when I re-exported a CSV of the blacklist, those email addresses weren't removed. Similarly, I could re-run my script, and the API would once again report it'd successfully deleted the email addresses, so clearly they were hanging around.
Ultimately, this is an AOL problem, not a Mandrill problem. You'll hit the Mandrill blocklist, but even if you remove the AOL email from Mandrill's blacklist, AOL will still bounce the message. Ask the user to add your sending email address to their contacts, and then remove their address from Mandrill blacklist and mark the account as valid in Xenforo. It's worked for my sites.
Last edited:
nrep
Well-known member
Good work @jeffwidman - I hope Mandrill take your bug reports onboard and come out with a fix soon. They've got a great platform, but there are some problems that make it difficult than necessary to work with.
Please do let us know if you make any progress
.
Please do let us know if you make any progress
.
Similar threads
- Question
