I think we might need to see a copy of your IPB data (and reproduce it with the data set). I don't recall this being reported before and there are no differences in the import code for forums between the IPB 3 and 4 importers (which could explain the problem). Is this viable?
Note though that if we do identify a bug, it will require a fresh import to resolve (which would mean a fresh install of XenForo/restore to pre-import state and losing any customizations done or data added since).