SparkPost Mail Transport for XF 2.2

SparkPost Mail Transport for XF 2.2 2.1.4

No permission to download
Darn microsoft outlook has a huge delay for some reason.

If your SparkPost account is new without a good sending reputation - or if you're using a shared IP address that doesn't have good reputation, then sending can often be delayed by various ESPs as an anti-spam measure on their part.

Microsoft is one of the worst at doing this.

If you log into your SparkPost account, go to Signals Analytics; Analytics Report; and then look at the Delayed Report preset, it will tell you how many emails are being delayed and for what reason.
 
If your SparkPost account is new without a good sending reputation - or if you're using a shared IP address that doesn't have good reputation, then sending can often be delayed by various ESPs as an anti-spam measure on their part.

Microsoft is one of the worst at doing this.

If you log into your SparkPost account, go to Signals Analytics; Analytics Report; and then look at the Delayed Report preset, it will tell you how many emails are being delayed and for what reason.
Thanks for the advice Sim. I gotta buy you a beer some time lol.

Gonna take a look at those when I get to my desk soon!
 
How many times have you received this error?

It shouldn't occur during normal operation - the XF cron task for fetching message events doesn't run frequently enough to cause this issue.

What can happen is that if it is run multiple times in quick succession, you'll receive an error because the SparkPost API is a bit fussy about the "from" and "to" dates being too close together (even if they are actually a second apart!!). So if you had a backlog of XF jobs waiting to be processed, it might happen - but should not happen once the backlog is cleared.

So if it only happened when you first changed to the server based trigger and hasn't happened since, I wouldn't worry about it.

However, if you do continue to receive the error, please do let me know!
Hi Sim,
It turns out when I changed to server trigger for cronjobs it wasn't working and not running at all. Hence I stopped getting the errors.

But I went back to making the cronjobs work and it's now again spewing these errors (so far it happened 3 times in a day [random times throughout]). I'm monitoring it right now but it's a pretty harmless error from what you're describing right? Just a situation where it's being run multiple times? Btw, what is the cronjob task exactly that is causing this error?
 
Hi Sim,
It turns out when I changed to server trigger for cronjobs it wasn't working and not running at all. Hence I stopped getting the errors.

But I went back to making the cronjobs work and it's now again spewing these errors (so far it happened 3 times in a day [random times throughout]). I'm monitoring it right now but it's a pretty harmless error from what you're describing right? Just a situation where it's being run multiple times? Btw, what is the cronjob task exactly that is causing this error?

The cron task is called "SparkPostMail: Fetch Message Events" - it communicates with the SparkPost API to fetch all bounce, spam complaint, unsubscribe, etc type messages.

Keep monitoring and let me know if it continues to generate new error messages - post a copy of the server error log for a new error (one that has happened since I posted this message), and I'll investigate.
 
The cron task is called "SparkPostMail: Fetch Message Events" - it communicates with the SparkPost API to fetch all bounce, spam complaint, unsubscribe, etc type messages.

Keep monitoring and let me know if it continues to generate new error messages - post a copy of the server error log for a new error (one that has happened since I posted this message), and I'll investigate.
Here's a new one that popped up today. Yesterday, had no issues.
Code:
[LIST]
[*]SparkPost\SparkPostException: Job Hampel\SparkPostMail:MessageEvent: {"errors":[{"param":"from","message":"from must be before to","value":"2022-09-26T15:08:00"}]}
[*]src/addons/Hampel/SparkPostMail/vendor/sparkpost/sparkpost/lib/SparkPost/SparkPostPromise.php:77
[*]Generated by: Unknown account
[*]Sep 26, 2022 at 8:08 AM
[/LIST]
[HEADING=2]Stack trace[/HEADING]
#0 src/addons/Hampel/SparkPostMail/SubContainer/SparkPost.php(76): SparkPost\SparkPostPromise->wait()
#1 src/addons/Hampel/SparkPostMail/Job/MessageEvent.php(50): Hampel\SparkPostMail\SubContainer\SparkPost->getMessageEvents(1, 1000, Array, 1664204899, 1664204908)
#2 src/XF/Job/Manager.php(260): Hampel\SparkPostMail\Job\MessageEvent->run(7.98808)
#3 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 7.98808)
#4 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 7.98808)
#5 job.php(43): XF\Job\Manager->runQueue(false, 8)
#6 {main}

-------------

Previous GuzzleHttp\Exception\ClientException: Client error: `GET https://api.sparkpost.com/api/v1/events/message?page=1&per_page=1000&events=bounce,policy_rejection,out_of_band,generation_rejection,spam_complaint,list_unsubscribe,link_unsubscribe&from=2022-09-26T15%3A08&to=2022-09-26T15%3A08` resulted in a `400 Bad Request` response:
{"errors":[{"param":"from","message":"from must be before to","value":"2022-09-26T15:08:00"}]}
 - src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113
#0 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Middleware.php(65): GuzzleHttp\Exception\RequestException::create(Object(GuzzleHttp\Psr7\Request), Object(GuzzleHttp\Psr7\Response))
#1 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Response))
#2 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), NULL)
#3 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}()
#4 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(118): GuzzleHttp\Promise\TaskQueue->run()
#5 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(145): GuzzleHttp\Handler\CurlMultiHandler->tick()
#6 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\Handler\CurlMultiHandler->execute(true)
#7 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\Promise\Promise->invokeWaitFn()
#8 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\Promise\Promise->waitIfPending()
#9 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\Promise\Promise->invokeWaitList()
#10 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
#11 src/addons/Hampel/SparkPostMail/vendor/php-http/guzzle6-adapter/src/Promise.php(94): GuzzleHttp\Promise\Promise->wait(false)
#12 src/addons/Hampel/SparkPostMail/vendor/sparkpost/sparkpost/lib/SparkPost/SparkPostPromise.php(73): Http\Adapter\Guzzle6\Promise->wait(true)
#13 src/addons/Hampel/SparkPostMail/SubContainer/SparkPost.php(76): SparkPost\SparkPostPromise->wait()
#14 src/addons/Hampel/SparkPostMail/Job/MessageEvent.php(50): Hampel\SparkPostMail\SubContainer\SparkPost->getMessageEvents(1, 1000, Array, 1664204899, 1664204908)
#15 src/XF/Job/Manager.php(260): Hampel\SparkPostMail\Job\MessageEvent->run(7.98808)
#16 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 7.98808)
#17 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 7.98808)
#18 job.php(43): XF\Job\Manager->runQueue(false, 8)
#19 {main}
[HEADING=2]Request state[/HEADING]
array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(40) "*removed*.com/admin.php?add-ons/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
 
@John917 I've released v2.1.4 which should hopefully avoid these API errors now.
Hey Sim,
Still getting the error a couple of times. I only had this one saved below as I was doing some dev work and accidentally deleted the other logs.

Code:
[LIST]
[*]SparkPost\SparkPostException: Job Hampel\SparkPostMail:MessageEvent: {"errors":[{"param":"from","message":"from must be before to","value":"2022-10-01T05:51:00"}]}
[*]src/addons/Hampel/SparkPostMail/vendor/sparkpost/sparkpost/lib/SparkPost/SparkPostPromise.php:77
[*]Generated by: Unknown account
[*]Sep 30, 2022 at 10:51 PM
[/LIST]
[HEADING=2]Stack trace[/HEADING]
#0 src/addons/Hampel/SparkPostMail/SubContainer/SparkPost.php(76): SparkPost\SparkPostPromise->wait()
#1 src/addons/Hampel/SparkPostMail/Job/MessageEvent.php(50): Hampel\SparkPostMail\SubContainer\SparkPost->getMessageEvents(1, 1000, Array, 1664603513, 1664603516)
#2 src/XF/Job/Manager.php(260): Hampel\SparkPostMail\Job\MessageEvent->run(5.22338)
#3 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 5.22338)
#4 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 5.22338)
#5 job.php(43): XF\Job\Manager->runQueue(false, 8)
#6 {main}

-------------

Previous GuzzleHttp\Exception\ClientException: Client error: `GET https://api.sparkpost.com/api/v1/events/message?page=1&per_page=1000&events=bounce,policy_rejection,out_of_band,generation_rejection,spam_complaint,list_unsubscribe,link_unsubscribe&from=2022-10-01T05%3A51&to=2022-10-01T05%3A51` resulted in a `400 Bad Request` response:
{"errors":[{"param":"from","message":"from must be before to","value":"2022-10-01T05:51:00"}]}
 - src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113
#0 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Middleware.php(65): GuzzleHttp\Exception\RequestException::create(Object(GuzzleHttp\Psr7\Request), Object(GuzzleHttp\Psr7\Response))
#1 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(204): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Response))
#2 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(153): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), NULL)
#3 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/TaskQueue.php(48): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}()
#4 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(118): GuzzleHttp\Promise\TaskQueue->run()
#5 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(145): GuzzleHttp\Handler\CurlMultiHandler->tick()
#6 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(248): GuzzleHttp\Handler\CurlMultiHandler->execute(true)
#7 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(224): GuzzleHttp\Promise\Promise->invokeWaitFn()
#8 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(269): GuzzleHttp\Promise\Promise->waitIfPending()
#9 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(226): GuzzleHttp\Promise\Promise->invokeWaitList()
#10 src/addons/Hampel/SparkPostMail/vendor/guzzlehttp/promises/src/Promise.php(62): GuzzleHttp\Promise\Promise->waitIfPending()
#11 src/addons/Hampel/SparkPostMail/vendor/php-http/guzzle6-adapter/src/Promise.php(94): GuzzleHttp\Promise\Promise->wait(false)
#12 src/addons/Hampel/SparkPostMail/vendor/sparkpost/sparkpost/lib/SparkPost/SparkPostPromise.php(73): Http\Adapter\Guzzle6\Promise->wait(true)
#13 src/addons/Hampel/SparkPostMail/SubContainer/SparkPost.php(76): SparkPost\SparkPostPromise->wait()
#14 src/addons/Hampel/SparkPostMail/Job/MessageEvent.php(50): Hampel\SparkPostMail\SubContainer\SparkPost->getMessageEvents(1, 1000, Array, 1664603513, 1664603516)
#15 src/XF/Job/Manager.php(260): Hampel\SparkPostMail\Job\MessageEvent->run(5.22338)
#16 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 5.22338)
#17 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 5.22338)
#18 job.php(43): XF\Job\Manager->runQueue(false, 8)
#19 {main}
[HEADING=2]Request state[/HEADING]
array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(28) "https://website.com/forums/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
 
What is your dev server setup? I suspect there's something messed up with the server time - this should not happen. Either that or you haven't deployed the new version correctly?

From your logs, you can clearly see the call SparkPost->getMessageEvents(1, 1000, Array, 1664603513, 1664603516) - and there's less than 3 seconds between the "from" time and the "to" time, which cannot happen in my latest version.

Can you check the actual code that's on your server?

Look in <server root>/src/addons/Hampel/SparkPostMail/Job/MessageEvent.php at lines 32-36 ... if it doesn't contain the following, then you're still using the old code:

PHP:
                if (\XF::$time - $from < 60)
                {
                    // sanity checking, start time should not be later than end time
                    $from = \XF::$time - 60;
                }
 
What is your dev server setup? I suspect there's something messed up with the server time - this should not happen. Either that or you haven't deployed the new version correctly?

From your logs, you can clearly see the call SparkPost->getMessageEvents(1, 1000, Array, 1664603513, 1664603516) - and there's less than 3 seconds between the "from" time and the "to" time, which cannot happen in my latest version.

Can you check the actual code that's on your server?

Look in <server root>/src/addons/Hampel/SparkPostMail/Job/MessageEvent.php at lines 32-36 ... if it doesn't contain the following, then you're still using the old code:

PHP:
                if (\XF::$time - $from < 60)
                {
                    // sanity checking, start time should not be later than end time
                    $from = \XF::$time - 60;
                }
Sorry I took so long I was swamped with work.
I swear I updated but turns out I had the wrong version.
😭
 
  • Like
Reactions: Sim
@Sim I just migrated to your add-on and it's great. I uploaded my suppressions from Mailgun over to Sparkpost. Is there anyway I can make the add-on turn all of these to "Email invalid (bounced)" without manually changing them all?
 
@Sim I just migrated to your add-on and it's great. I uploaded my suppressions from Mailgun over to Sparkpost. Is there anyway I can make the add-on turn all of these to "Email invalid (bounced)" without manually changing them all?

Hi @JBS - I don't really think you need to worry about that.

If your forum tries to send an email to someone in the suppression list, it will bounce back with an "admin" failure - which are treated as hard bounces by the addon and will automatically see their user status changed to "Email invalid (bounced)".
 
  • Like
Reactions: JBS
Thank you so much for this incredible add-on, Sim!

If a user is on a suppression list, you will detect the bounce and change user status to "Email invalid (bounced)". Can I ask how through the revalidation process the user is removed from the list so emails can eventually get through? I don't see any mention of the suppression list API in the code. Thanks!
 
Thank you so much for this incredible add-on, Sim!

If a user is on a suppression list, you will detect the bounce and change user status to "Email invalid (bounced)". Can I ask how through the revalidation process the user is removed from the list so emails can eventually get through? I don't see any mention of the suppression list API in the code. Thanks!

I have a pro version of the addon in development which allows you to remove someone from the suppression list - but that addon won't be released for quite a long time though - have higher priorities at the moment.

In the meantime, you can do this manually by logging into the SparkPost UI - https://app.sparkpost.com/lists/suppressions

My recommendation is to perform the following tasks:
  1. remove the user from the suppression list manually using the SparkPost UI
  2. edit the user profile and set their user state to "Awaiting email confirmation (from edit)"
  3. resend the account confirmation from the Actions menu when editing the user profile.
It's important that you remove the user from the suppression list first, otherwise the account confirmation email won't get through.
 
I look forward to being your first customer for the pro version.

I actually already have a tool for members to use to get themselves off the suppression list, I will integrate that into the re-confirmation flow communications. I have a user right now who is stuck in a loop where the re-confirmation emails are bouncing due to them having got onto the transactional suppression list due to "reason: spam complaint". I've noticed this happening a lot over the years with various mail hosts having confusing buttons or messaging for users to interact with that transmit that back to Sparkpost and put them on the suppression list. I think it would be great if the addon removed the user from the suppression list prior to sending the reconfirmation email.
 
Hi! Suggestion and request to update dependencies (old versions of Guzzle, etc. are used) and compatibility with XF 2.3

This will require a complete rewrite for XF 2.3 given the change from SwiftMailer to Symfony Mail - it's on my todo list, but will take a while for me to get to it.

I'll aim to have a version working before XF 2.3 comes out of beta.
 
Top Bottom