Yeah it's a bit of a design issue but one that's mostly necessary. The image itself isn't stored twice, but the thumbnail is. One is created by the XF attachment system, the other is created for the Gallery.
As it is only the thumbnails it is safe to ignore as the space requirements are less than the full size images.
Stored on the server. Full size images and attachments are in the internal_data/attachments directory. In the numbered directories you will find the .data files. These are the original files for the gallery or uploaded attachments.