What @DarkGizmo said.
I've currently got 19 add-ons (I've had more in the past). Some people have less. A few people have none. Some people have dozens.
It'll all come down to what you require for your site, whether or not it's a core feature of XenForo and if not, whether there's an add-on to fill the gap.
The amount of add-ons varies from forum owner to forum owner. It depends on which you need and what you want to accomplish with your site.
+ how you config the server.performance is impacted depends on the add-on, what it does, and how it has been coded.