How are you determining there are bounce entries missed? If you're accessing the account, you may be making them unavailable to XF to process (either because they're not available via POP3 or because they're not considered new via IMAP). In general, you won't want to access the account if there are pending messages (though the specifics will vary based on your mail server).
The bounce log will include every email it reads, even if it can't find relevant details. So if you're not checking the account, that would potentially imply an issue reading the account or an error (a fatal one?) when reading the messages. I assume there are no server error logs in XF, so an error would have to either be a fatal PHP error or something killing the process.