What server errors have you received?
As for the bounced emails, if you are seeing them within the bounced email log, then the system is working perfectly. It accesses the proper account, reads the emails, reacts and deletes them from the account. So manually checking the account will show an empty inbox.