Unless you write a PHP script to automate it, there's probably not a great deal you can do.
To automate it, it would require fetching media records, preferably in small batches, and attempting to retrieve the attachment data for each one. You would have to take care that this only targets media with the type "image_upload" and "video_upload" as obviously video embeds don't have attachments.
Once you locate a media item that doesn't have an attachment, I recommend using the XF DataWrtiter system to programmatically delete the entry. This updates various counts and other caches that may be relevant.
If you were doing it manually, I'd recommend doing it via the Admin CP > Applications > Attachment Browser. This can be filtered by content type so that might make things easier to find.
In fact, if you deleted the contents of the data/attachments directory, and then rebuilt the attachment thumbs (Admin CP > Tools > Rebuild Caches > Rebuild Attachment Thumbnails), you will be able to more easily identify the broken images by those that have broken thumbnails in this list.
From there you can "View host content" and then delete the media item in the proper way.
