SMTP is only for sending (but it will work while using SMTP). The bounce processing needs to be setup separately.
For Mandrill, this involves setting the option in their control panel to forward bounce emails to an email address, and then setting up XenForo to access that email address in the ACP.
You should use an email address specifically for bounce processing (ie that doesn't receive other mail).