1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Import speed drops dramatically on Attachments

Discussion in 'Installation, Upgrade, and Import Support' started by Baron, Jan 3, 2013.

  1. Baron

    Baron Member

    I'm migrating my fairly large forum to xenforo from a vb 3.8 database and everything goes fine until I get to the attachments import, then it's painfully slow. There are about 175,000 attachments in the database and the importer is processing those at about 20% per DAY! At this rate, it will take 5 days to import, which doesn't seem right. Perhaps my DB settings aren't optimized for handling these types of transactions?

    My server is an Apple Xserve with 2 x 3.0 GHz Dual-Core Intel Xeon processors and 8GB of Ram

    Attached is my MySQL variable output....

    Any insight is greatly appreciated!

    Attached Files:

  2. Slavik

    Slavik XenForo Moderator Staff Member

    That is a lot of attachments!

    Potentially just a bottleneck on I/O resources.

    How large is your vb3 database?
    Jake Bunce likes this.
  3. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    Are the vB3 attachments stored in the database or file system? I would hope they are in the file system.

    Increasing the batch size might help:


    There is also the command line importer which would be ideal, but it's not officially supported and people have reported problems with it.

    And as Slavik said, I/O could be a problem here. Importing 175k attachments is going to involve a lot of I/O. If you can improve your disk performance or even install a SSD you could benefit greatly.
  4. Baron

    Baron Member

    They are in the database :eek:

    This import is a test run so do you think it would be a better strategy to first have vb 3.8 convert the attachments to the filesystem and then do the xf import?
  5. Slavik

    Slavik XenForo Moderator Staff Member

    It would be worth a shot.
  6. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    The problem is that in vB3 the actual file data was still stored in the attachment table instead of in a separate table = crappy performance when you have lots of attachments.

  7. Baron

    Baron Member

    So just as a followup, I used VB's attachment manager to dump the attachments out of the database and into the filesystem. When I retried the attachment import into XF, my import time went from 5 days to an hour and a half! :D
    Jake Bunce, MattW and Slavik like this.
  8. Slavik

    Slavik XenForo Moderator Staff Member

    Good to hear :)

Share This Page