VBulletin Migration Performance

[JamesE]

Hi everyone,

I am currently doing a Vbulletin 3 to Xenforo migration, with approximately 430,000 posts and around 15GB of VB attachments.

How long would you expect a conversion of this size to take? We are currently facing a 6-8 hour migration cycle - this seems to be to be overly large and think it's the attachment conversions that are taking the time. If this is way more than expected, where can we see what is taking up the bulk of the time and how can we mitigate it?

Thanks

 

[Slavik]

Hi everyone,

I am currently doing a Vbulletin 3 to Xenforo migration, with approximately 430,000 posts and around 15GB of VB attachments.

How long would you expect a conversion of this size to take? We are currently facing a 6-8 hour migration cycle - this seems to be to be overly large and think it's the attachment conversions that are taking the time. If this is way more than expected, where can we see what is taking up the bulk of the time and how can we mitigate it?

Thanks

Retain content ID's.

Do a test import.

Export the attachment tables, (and data/internal data dirs if your wiping clean)

Do live import, also retaining content IDs, skip attachments.

Import your previously exported attachment tables/folders

Complete import

 

[Mike]

I suspect it is the attachments. Attachments create a lot of work, especially with things like thumbnails being generated. As such, it doesn't seem like it's orders of magnitude out or anything like that.

 

[JamesE]

Hey guys,

I'm confused between the responses of Slavik and Mike.

Can Slavik please explain by what you mean with your exports and imports and the retaining of the ContentIDs? It sounds like what you are saying is that the import/export is as simple as:
- Retaining the ContentIDs - I assume this means that Xenforo will use the old VB identifiers for the attachments?
A Test Import (does this create all the DB entries without making modifications to the files? And it keeps the old VBulletin file identifiers?)
- Export the MYSQL Attachment tables and copy the data/internal data files
- Live Import, which does the import of all the other data
- Replace the DB replaces from the backups, and replace the internal_data files

There is about 50,000 VB attachments to push in. What is Mike referring to when he says that the attachments will create a lot of work then?

 

[Paul B]

Bear in mind, if you do a partial restore of the database (the attachment tables) from an earlier import, any changes made to the live site with regards to attachments between doing the test and final import, will not be reflected in those backed up tables and could result in corruption and other problems.

 

[Mike]

Slavik is roughly presenting an alternative way to save time by effectively doing the attachment step earlier and restoring the previous results for the live import, though it wouldn't be a recommended/supported approach. Personally, I would only recommend doing a "full" import.

There is about 50,000 VB attachments to push in. What is Mike referring to when he says that the attachments will create a lot of work then?

Yes. Every image will need to be processed and thumbnails created, for example. All the data needs to be copied about.

When you do a test import, you should be able to see how long each step takes, BTW.

 

[Slavik]

Err yes sorry I thought this was a conversation reply as I received a conversation asking almost the exact identical thing.

However, based on my experience, 6-8 hours for a board and that sort of attachment count would be about right.

 

[PASS]

I'm going to be doing the same thing soon. Should I use imagemagic or GD?

 

[Slavik]

I'm going to be doing the same thing soon. Should I use imagemagic or GD?

Whichever you prefer.