XF 2.3 vBulletin import VERY SLOW.

[iMav]

Importing from vBulletin 4.1.7. Using the web import (didn’t realize that CLI might be a better option…and can’t switch without starting over).

We are going on 3+ days. Currently importing posts (just shy of 70%). It’s been doing this step for a good 30 hours or so.

Core i7 with 64GB ram. Mirrored spinning disks, so maybe some performance issues there. Any tweaks I can make to help speed things up?

 

[Digital Doctor]

Forum stats ?

 

[iMav]

Threads: 182,019
Posts: 2,868,672
Members: 15,575

 

[iMav]

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

 

[iMav]

If the CLI import is an order of magnitude (or two) faster, I suppose I can wipe the new forum and have a fresh go at it.

It doesn't seem to have run into any errors/issues with the import. Just. SLOW.

 

[iMav]

I'll be on a flight monday afternoon. I'd REALLY like to have the import at least done by then. Otherwise, since it's not a CLI running on a screen session, dealing with this when I get to my hotel will be a pain. I'd have to set up a remote VM with a browser so I can let it continue to run without having to stop and restart on my laptop as I go from place to place. (Never in my dreams did I think it would take close to this long. Migrating geekhack from vB to SMF was WAY faster...and a significantly bigger community.)

 

[iMav]

Ok. I bit the bullet. Cut my losses and am doing a fresh install and will initiate the import from the CLI (in a screen session).

At least I can more easily babysit it remotely if it is still super slow.

(The sunk cost fallacy was really burdening me. )

 

[MySiteGuy]

Its really not a large enough forum that even with browser mode it should take that long if MySQL is tuned and not using an out of the box configuration (which is terrible). Even rust drives are plenty fast for imports when MySQL is tuned.

 

[iMav]

Well, I haven’t done any substantive tuning to MySQL. Any specific recommendations? (Based on my stated forum size, server stats, and knowing this is a single purpose install.)

Still not quick, but significantly faster. (Several hours to get to step 16 / posts instead of days.)

 

[iMav]

Still grinding away. Looks like the posts will be taking around 24 hours.

 

[philmckrackon]

That is slow, you should take @MySiteGuy advice and optimized MySQL.

 

[iMav]

That is slow, you should take @MySiteGuy advice and optimized MySQL.

Definitely open to best practices suggestions.
(and, obviously, if there are things that can be done that don't require to start over again, that would be optimal... However, if needed, I can revert to a fresh install and re-start the import process.)

 

[iMav]

I've seen reference to Centminmod. I'm running on the latest LTS version of Ubuntu Server, which does not seem to be supported. I am leveraging a LEMP stack (have been an nginx guy for quite a while).

I guess I am fine with letting this import take however long it takes at this point (since I am doing it via CLI in a screen session...so, easy to babysit while traveling). If there are some good 80-20 suggestions to get things a bit more optimized, I would love to start there. (and then tweak more once the nature of the usage on the new server is more clear)

 

[ehd]

In my experience NVMes have so many more IOPS. Harddisks might be the bottleneck which slows everything down.

Saw this very cleary when switching to ssd/nvme as server-storage.

 

[iMav]

In my experience NVMes have so many more IOPS. Harddisks might be the bottleneck which slows everything down.

Saw this very cleary when switching to ssd/nvme as server-storage.

Yeah. I'll have to see how the community performs once the migration is complete. (I can take a slow migration if the day-to-day usage is fine performance-wise.)

I did make a few tweaks based on the "mysqltuner" perl script. Seems to have helped a bit.