• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Import speed drops dramatically on Attachments

#1
I'm migrating my fairly large forum to xenforo from a vb 3.8 database and everything goes fine until I get to the attachments import, then it's painfully slow. There are about 175,000 attachments in the database and the importer is processing those at about 20% per DAY! At this rate, it will take 5 days to import, which doesn't seem right. Perhaps my DB settings aren't optimized for handling these types of transactions?

My server is an Apple Xserve with 2 x 3.0 GHz Dual-Core Intel Xeon processors and 8GB of Ram

Attached is my MySQL variable output....

Any insight is greatly appreciated!
 

Attachments

Jake Bunce

XenForo moderator
Staff member
#3
Are the vB3 attachments stored in the database or file system? I would hope they are in the file system.

Increasing the batch size might help:

http://xenforo.com/community/thread...port-be-can-we-speed-it-up.35324/#post-401122

There is also the command line importer which would be ideal, but it's not officially supported and people have reported problems with it.

And as Slavik said, I/O could be a problem here. Importing 175k attachments is going to involve a lot of I/O. If you can improve your disk performance or even install a SSD you could benefit greatly.
 
#4
Are the vB3 attachments stored in the database or file system? I would hope they are in the file system.
They are in the database :eek:

This import is a test run so do you think it would be a better strategy to first have vb 3.8 convert the attachments to the filesystem and then do the xf import?
 

Slavik

XenForo moderator
Staff member
#5
They are in the database :eek:

This import is a test run so do you think it would be a better strategy to first have vb 3.8 convert the attachments to the filesystem and then do the xf import?
It would be worth a shot.
 
#7
So just as a followup, I used VB's attachment manager to dump the attachments out of the database and into the filesystem. When I retried the attachment import into XF, my import time went from 5 days to an hour and a half! :D
 

Slavik

XenForo moderator
Staff member
#8
So just as a followup, I used VB's attachment manager to dump the attachments out of the database and into the filesystem. When I retried the attachment import into XF, my import time went from 5 days to an hour and a half! :D
Good to hear :)