If you have a member with zero posts that have been deleted, they will be removed from a poll. You will see a space where they were removed, surrounded by commas. If you look at the underlying source code, there will be a link to their profile if you view source. This results in a 404 error. This happens even after rebuilding poll cache.
When you say "You will see a space where they were removed", where are you seeing this? I'm assuming that you mean in the list of names in the overlay that appears when you click the vote count on the results of a public poll?
If so, I don't get the empty-space-surrounded-by-commas issue unless I run the poll cache rebuilder, but the behaviour before I do that, where the name is retained but leads to a 'member not found' error, in the same way as would happen with a guest, would be the intended behaviour.
What I may look into doing is altering the behaviour of the poll cache rebuilder so that it retains the name of the deleted user for historical record purposes.