Feedback: conversion of a vBulletin forum with 2.8 million posts

Walter

Well-known member
For testing purposes I converted a forum with 2.8 million posts from vBulletin 3.8 to Xenforo.

For a forum of that size I was interested in seeing four things:
  1. If the conversion went well and
  2. how long it took,
  3. what the forums database size would be and
  4. if search was ok without using any hacks or Sphinx
Here are my experiences: while Beta 1 had troubles converting the users Beta 2 seemed to work fine, but it failed later while converting the forum posts. Fortunately Mike was so kind to post a fix for that here and the import finally worked and my questions are all answered:

  1. Everything was converted well, at least everything I expected (of course blog and posts in social groups are currently not supported). Even umlauts :)
  2. Due to the problems during converting the posts I don't have exact times but the import took under 24 hours. After that I rebuilt the search index which took only a few hours (about 11 pm to maybe 7 am) - that is quite fast considering that it created a search index table with 6 GB of data!
  3. Database size is quite larger, vBulletin was at 6.1 GB in 202 tables and XenForo is at 9.4 GB in 119 tables. Most of the 9.4 GB is used by the search index, an astonishing 6 GB.
  4. I made no scientific tests but playing with the search function and trying different kinds of searches (e.g. posts or titles only) it seems that XenForo is at least as fast as vBulletin 3.8 (stock vb, no Sphinx or other hacks). I guess I can improve search speed further with tuning mySQL specifically for Xenforo.
I hope that this is of some help. The converter seems quite stable (at least with the fix from above applied). There are still a few features missing before I can convert that particular forum (an alternative for the blogs and heavily used social groups, a picture gallery) but it's quite impressive what Kier and Mike have delivered.

Regarding the quite fast import: please consider that this was done with a server with a very good disk system (8x SAS RAID10) which is a major factor. CPU isn't such a big concern (server had only a single Xeon 3220). Nevertheless, the import was done while the server was under load. Having a good disk subsystem and optimized mySQL goes far.
 
For testing purposes I converted a forum with 2.8 million posts from vBulletin 3.8 to Xenforo.

For a forum of that size I was interested in seeing four things:
  1. If the conversion went well and
  2. how long it took,
  3. what the forums database size would be and
  4. if search was ok without using any hacks or Sphinx
Here are my experiences: while Beta 1 had troubles converting the users Beta 2 seemed to work fine, but it failed later while converting the forum posts. Fortunately Mike was so kind to post a fix for that here and the import finally worked and my questions are all answered:

  1. Everything was converted well, at least everything I expected (of course blog and posts in social groups are currently not supported). Even umlauts :)
  2. Due to the problems during converting the posts I don't have exact times but the import took under 24 hours. After that I rebuilt the search index which took only a few hours (about 11 pm to maybe 7 am) - that is quite fast considering that it created a search index table with 6 GB of data!
  3. Database size is quite larger, vBulletin was at 6.1 GB in 202 tables and XenForo is at 9.4 GB in 119 tables. Most of the 9.4 GB is used by the search index, an astonishing 6 GB.
  4. I made no scientific tests but playing with the search function and trying different kinds of searches (e.g. posts or titles only) it seems that XenForo is at least as fast as vBulletin 3.8 (stock vb, no Sphinx or other hacks). I guess I can improve search speed further with tuning mySQL specifically for Xenforo.
I hope that this is of some help. The converter seems quite stable (at least with the fix from above applied). There are still a few features missing before I can convert that particular forum (an alternative for the blogs and heavily used social groups, a picture gallery) but it's quite impressive what Kier and Mike have delivered.

Regarding the quite fast import: please consider that this was done with a server with a very good disk system (8x SAS RAID10) which is a major factor. CPU isn't such a big concern (server had only a single Xeon 3220). Nevertheless, the import was done while the server was under load. Having a good disk subsystem and optimized mySQL goes far.
what mysql optimizations did you perform?
 
Currently no Xenforo specific ones as I still don't know XF good enough.
The other optimizations are the usual ones: try to keep tmp tables in memory, sufficient key buffer, table cache,...
 
Definitely helps because this way you can optimize one server for web and one fot the DB and don't have to make compromizes. An extra database server usually is able to keep either all or at least the most important part of the db in memory which speeds things up dramatically.
 
Top Bottom