It is a LIFESAVER! We have it set so that any 3 different members that report the same post, that post automatically becomes hidden for a moderator to review.
Sure there is potential for abuse, but like most features that's always a possibility. In our cases, the definite pros of having this feature outweigh the potential cons, especially when porn spammers get on the forum in the middle of the night and no moderators are on to remove the content.
In our communities we've only seen potential abuse once, and we moderated those few people and told them if we find they area abusing the feature they will bet banned. That took care of the problem really quick.
My other forum software doesn't support "hidden" posts or "held for moderation" posts, so the text of the post is replaced by a message like "This post is being held for moderation". I guess in xF this isn't necessary since xF supports hidden posts?
The main option in plugin: "How many unique member reports before the post is held for moderation?"
I don't expect this to be added to the core, but I thought it might be a great addon. Heck I'd even be willing to pay to have the addon made, or contribute to it's development depending on the final cost.