XF 1.2 Import speed - how long should it take, and can I make it faster?

[AntonyF]

Hi,

I'm importing from a vBulletin 3.8.6 system to XenForo 1.2. 1.2m posts in total.

The thing is, it seems to be taking a long time. Long time is of course relative. But I seem to feel a few years ago when I did a test import that it was a lot faster. Plus my new server is a lot faster... quad core I believe, plenty of RAM (sorry stats escape me). I also remember way back kin the day when importing *into* vBulletin some mySQL config changes could make a world of difference.

So:
1) How long do you think it should take to import 1.2m posts?
2) Any tips on what to tweak to speed up the mport?

The only change tried was editing the config from a limit of 100 to 1000 as per a thread I saw somewhere, but not sure if that's helped or is even advisable.

Thanks
Antony

 

[Paul B]

How long is it actually taking.
What sort of times are you seeing for each step?

 

[Mike]

MySQL and I/O tuning is by far the best thing you could do. Your MySQL DB focus should be InnoDB.

(I have seen huge performance increases disabling barriers in an ext4 file system, but that's not recommended and may be dangerous in a crash. If your server isn't doing anything other than the import though, that may be an option while the import runs.)

 

[AntonyF]

Well I'm importing posts right now. I think it's probably done 6 - 8 hours, but not sure. It started at 70 something percent, which confused me. I just watched it and it's incrementing 0.01% about every 50 seconds to a minute. So at it's on less than 80%, that's gonna take quite some time!

Mike any particular quick fix suggestions? I used to tamper with mySQL a lot a few years back when vBulletin kept falling over. Now I have a new server which works too well, so I've done zilch and got out of touch.

Thanks
Antony

 

[Mike]

These are the sorts of pages you'd want to look at to start:

http://www.mysqlperformanceblog.com/2007/11/01/innodb-performance-optimization-basics/
http://dev.mysql.com/doc/refman/5.0/en/innodb-tuning.html

MySQL/server tuning is a very detailed thing and it depends on all sorts of server things so it's not really possible to make one-size fits all recommendations. You can look at those links for some baseline recommendations though.

 

[AntonyF]

I remember changing one setting back in the day when importing to vBulletin and it made a huge difference... it's all foggy now, those braincells met a beer-fueled death long ago no doubt.

*sighs* Remind me why I want to fix what's not broken?

BTW how long would you expect it to take? Roughly? For 1.2m posts.

 

[CTXMedia]

Are there any indexes that could be dropped and rebuilt after the import?

 

[AntonyF]

Private messages import...

Imported 70,928 items. (2 hours 3 minutes 7.66 seconds)

Does that sound about right? I'm just trying to work out whether my import is slow or as expected.