XF sends that email immediately at which point it is no longer in our hands.
Slow delivery would usually be related to whatever method is used to send the email - in XF this is usually built-in sendmail or SMTP. You'll need to ascertain how your emails are sent and then seek support from whoever supports that in order to troubleshoot why the email delivery is slow.