Yeah... especially if you don't do it carefully... of course, some folks just get ban-happy (it's a power thing) and don't worry about people being able to have access and join.
I do blocking by ASN's for server access.. but even then you have to be careful because if you (or a user) has shared your content elsewhere, sometimes XF shows their server as a "guest" as they access that content... and it's easy to block that shared content (which you usually don't want to do as it can bring more users).