Would it be possible to have the mail submitted directly to Xenforo as an option to going through polling a mailbox?
For example on another project I am working on, Postfix will submit any emails it receives to a PHP script via an alias to a script which gets the email through file_get_contents("php://stdin"); That way you bypass the hassle of setting up a mail box and the constant polling required, which in my experience is often fraught with one complication or another, especially when dealing with large numbers of emails. It also takes out the reliance on a third party system.
The above is also a concern for me as when do email campaigns, we will send out tens of thousands of emails at once. In our setup we have one server which is always up and other servers which get created and destroyed according to demand. I think it would be best if it was just the 'always up' server which handles this workload.