Am I needlessly inconveniencing users?
Yes and no.
Technically speaking, you are always on the safe side when all of the attachments and images are hosted somewhere else. You have less liability by doing that.
But disallowing that wouldn't stop people in posting links to files/images which could be copyright infringement. That wouldn't be allowed, too (or gray area). Are you disabling link sharing for your site? I doubt it.
So in general, like here on XF, most boards allow attachments and images. I could upload right now a copyrighted material here in this post, if I wanted. But I am not allowed to do this. Reading the terms and rules will clarifiy it.
Even if I did and nobody noticed. People can ask for a takedown whenever they want or someone can file a DMCA takedown. Then the board owner must remove those links or files. Done.
Conversations are private anyway. So even if one shared such a file in private, who else would know about it but those participants and you as the board owner? Nobody. And even if people did know, you can always take it down as per request and you are fine.
So if I were you I would allow attachments for private conversations, if you also allow it for normal posts. Most board owners disable it not because of they are afraid of copyright infringement, but because of disk usage. It can blow up your disk usage when people can upload tons of megabytes.
TLDR
So, yes you are inconveniencing them but needlessly? No.
If I were you, I would allow them if you also allow them on board except you have a tight case of disk usage and you are on the verge of running out of space.
PS: If you live in the EU and the new articles 11 and 13 go through, forget everything what I said. Because then you will be liable for everything, no matter what.
