@AndyB has a couple of nice little addons that can help if you're interested.
Auto delete email confirm [Paid] will delete users who have not confirmed their email addresses after X days (selectable).
Auto delete rejected [Paid] will do the same thing for rejected registrations, again after a selectable time period.
It may not have much of an impact in terms of server/database space but it does help to trim the user list and make it easier to find things in the ACP.