limiting the followers (people who follow you) not a good idea, bcoz some popular person may get 10,000 followers who wanna get to know about his posts.
Its okay to limit the number of people one can follow. (following)
As others said adding load more.. button and adding 25 members at each load is the best solution. Its essential for large boards.