Removing the whitespaces will save you a total of 10kb traffic per siteload. Removing a single avatar/image from the site saves equal or even more. As said most people won't even recognize because when they try to inspect your code, they will most likely do it with Firefox or Chrome and then normally open the code inspector, which removes the whitespaces automatically anyway.
After all, every XenForo board is running with that much whitespace and I haven't seen a single one going down due to this. You're probably getting more serverload by removing it then by letting it be where it is.