The thing is, yes you can be held responsible. So it's a good idea to not allow copyright material and remove it when you find. Otherwise it's possible in some niches for your users to think there is no problem in doing it and before you know it the site is full of infringements and file sharing.
So it's kind of self proliferating. If your site is making money, I imagine that worse things than just DMCA takedown might happen.
OTOH, you don't want to stifle user activity. I tend to turn a blind eye to small things, but if I notice something with an obvious copyright (e.g. Getty Images watermark) then I remove it.
I tend to think Youtube embeds are OK, as Youtube should be policing those.