I don't know why someone should feel guilty. It's something each person is free to do.
I follow someone because I'm interested in what they do and say around the site, but they may not feel the same for me. It's just how things are, and if people feel guilty about it... I don't really know what to say to them.
Nope... no guilt here. I would even say its not an intentional thing, but more I read something I really like from someone in succession, so I click on follow to see what other great things they may or may not have to say.
This. I don't liken "following" somebody to "befriending" somebody (as is the feature in vBulletin). It's just a tool that allows members to follow members they find interesting, or whose posts they like, or whatever other reason...