XF 2.0 Importing from a large vBulletin database

[djbaxter]

I remember seeing something about this but can't find it.

I am considering migrating a vBulletin 4.2.5 forum with these stats:

Threads 60,236
Posts 9,021,738
Members 46,384

How easy is this to do and what issues should I anticipate?

 

[Vekseid]

Unless they've managed to bork things further from the 1.x importers (which are genuinely a mess), you should be able to setup a test install where you test the import function and see how it flows, what gotchas there are, anything that gets imported incorrectly, etc.

 

[djbaxter]

Yeah but that's not helpful. The owner is looking for some assurance BEFORE he buys Xenforo, instead of buying it and finding he's now got a big problem on his hands. I used the XF2 importer on a smaller vBulletin site and it worked flawlessly.

 

[aagcrc]

Hi, I just imported and switch my Forum last night from vBulletin 5.4.3 (the most terrible mistake to upgrade to VB5) to XenForo 2.0.1 using the official XenForo Importers 1.0.2 and all was straight forward.

Threads 376,480
Messages 4,873,846
Members 38,182

Took a total of 8 hours

I just had to tweak couple things, changing the DB char_set to utf8mb4.general_ci and change the old_id column lenght to char(150) in the new import_log table.

My users love it, specially with the speed, since VB5 it was just WAY TO SLOW.

 

[djbaxter]

Thank you. (I agree about vB5).

The forum I'd be converting has over 9 million posts so I should anticipate a wait. Did you use the command line or the browser version of the importer?

 

[Kirby]

Total size is usually not an issue, though it does take quite some time to import a forum with several millions of posts, especially if there are many attachments/photos as well.

If possible, I'd always use CLI in a screen session - this has way less overhead then using the browser.

So far I've not migrated any big forums (e.g > 10M posts), but those we did so far (1M UBB.threads with a custom importer class and 600K phpBB 3 via XF 1.5) went without major issues.

 

[Slavik]

Yeah but that's not helpful. The owner is looking for some assurance BEFORE he buys Xenforo, instead of buying it and finding he's now got a big problem on his hands. I used the XF2 importer on a smaller vBulletin site and it worked flawlessly.

Check my signature links, ive imported boards multiples larger than this fine, your biggest thing will likely be the wait time on the default importers.

 

[djbaxter]

Yes I'd definitely be using the command line version for this.

That's all promising - perhaps not the wait time but I can live with that.

 

[aagcrc]

Thank you. (I agree about vB5).

The forum I'd be converting has over 9 million posts so I should anticipate a wait. Did you use the command line or the browser version of the importer?

I start the import in the browser with all the config, then move to CLI option.


Import complete!
Your import is now complete and the necessary caches have been rebuilt.

Import log table
xf_import_log
This table contains mappings from old IDs to new IDs. You may need this for redirection scripts.

Total run time: 6.48 hours

Import totals
User groups:24 (1 seconds)Custom user fields:1 (0 seconds)Users:75,067 (9.08 minutes)Avatars:5,198 (6.72 minutes)Buddy and ignore lists:1,067 (4 seconds)Paid subscriptions:0Custom BB codes:3 (0 seconds)Private messages:100,826 (11.40 minutes)Profile comments:1,294 (5 seconds)Forums:88 (2 seconds)Moderators:1 (0 seconds)Thread prefixes:0RSS feed sources:0Threads:377,050 (15.27 minutes)Tags:4,363 (27 seconds)Posts:4,878,718 (6 hours, 40 minutes)Edit history:5,327 (47 seconds)Polls:1,429 (52 seconds)Attachments:50 (23 seconds)Positive reputation:9,002 (2.63 minutes)Infractions:4 (0 seconds)CAPTCHA Questions:2 (0 seconds)Custom FAQ pages:33 (4 seconds)Announcements:4 (1 seconds)Notices:0Smilies: 226 (1 seconds)

 

[aagcrc]

Yes I'd definitely be using the command line version for this.

That's all promising - perhaps not the wait time but I can live with that.

The stronger the server the faster. Also the config you could tweak php and mysql.

 

[djbaxter]

The stronger the server the faster. Also the config you could tweak php and mysql.

Is there a separate config file for that? or do you mean you set up your options when you run the importer?

 

[aagcrc]

When you install the importer add-on, you have to pre-configure the importer via Web Browser, then later go to the CLI and run the actual import based on the settings.

 

[LottoCED]

Hi,
I'm importing to forum from vbulletin 5 to xf 2 but the transfer time is really a lot.
Vbulletin's db is in the same xenforo server.
the server is a dedicated one with 32 GB ram, intel I7 CPU, centos6.
In the end I find this mistake.
you could give me indications of how to speed up the import ?

# /opt/plesk/php/7.3/bin/php cmd.php xf:import
Starting import from vBulletin 5.x (Beta)...
- Step  1 of 25: Gruppi utente             00:00:06 [14]
- Step  2 of 25: Campi personalizzati utente 00:00:00 [0]
- Step  3 of 25: Utenti                    00:31:59 [14.846]
- Step  4 of 25: Avatar                    00:02:16 [607]
- Step  5 of 25: Buddy and ignore lists    00:00:26 [104]
- Step  6 of 25: Paid subscriptions        00:00:00 [0]
- Step  7 of 25: Codici di formattazione personalizzati 00:00:00 [0]
- Step  8 of 25: Private messages          00:00:00 [6]
- Step  9 of 25: Profile comments          00:00:04 [43]
- Step 10 of 25: Indice del Forum          00:00:03 [22]
- Step 11 of 25: Moderatori                00:00:00 [0]
- Step 12 of 25: Prefissi discussioni      00:00:00 [0]
- Step 13 of 25: RSS feed sources          00:00:00 [0]
- Step 14 of 25: Discussioni               04:54:21 [144.141]
- Step 15 of 25: Messaggi                  64:48:57 [1.455.007] 90,64%

In Statement.php line 71:

  [E_WARNING] Error while sending STMT_EXECUTE packet. PID=9340


xf:import [--processes PROCESSES] [--finalize]