converting a huge community to xenforo

[shock]

my friend and me(i was the CEO) are curious about how much it will take to convert a community with this stats to xen foro:
users: 747,653.
messages: 83,613,000.
threads: 9,054,146.
forums: 938.(including subforums)
Average users online: 23,000.

it may be possible that the owner of the forum will consider switching the forum software, because VB is realy heavy, slow, and causes alot of problems with the search... (when users search alot the forum crashes)

and we wanna know if xen foro 1.1 can handle this stats with minimum troubles.

 

[Kier]

We've been working together with some very large boards to produce an importer that can take better advantage of server resources. In some cases it can be several time faster than the standard importer. It still needs work, but it's on the way...

 

[Marcus]

With your board and to avoid any speed issues / internal linking issues with redirecting, you can take a look at my idea to have your current IDs in place after your import:

http://xenforo.com/community/threads/import-with-the-same-ids-as-former-product.20028/

I have the same experience with importing like Floris, 1 Million post board took me around one hour.

 

[shock]

Ok, thanks to everyone, ill contact anyone how offered hes help if he(the owner) decide to use xenf.
right now it looks like the best option, but its alot of work, and we may reconsider converting the forum because of it(and bacause the forum is heavly moded).
again, thanks to everyone!

 

[Ryan McAdams]

Here are the stats from our 10m+ import.

More info including server specs etc can be found at http://tech.lazyllama.com/2011/08/08/converting-a-big-board-vbulletin-to-xenforo-migration/

What surprised me was that the live import wasn't much faster than the test despite vastly differing specs on the test and live server.

This is the same thing I saw... there wasn't any difference in speed from using an SSD which was weird

 

[Ryan McAdams]

We've been working together with some very large boards to produce an importer that can take better advantage of server resources. In some cases it can be several time faster than the standard importer. It still needs work, but it's on the way...

This is great news...

What might be nice Kier is if there was something completely out of browser for the larger boards that can be done from a command line perspective...

 

[CTXMedia]

What might be nice Kier is if there was something completely out of browser for the larger boards that can be done from a command line perspective...

Nah ... who wants to sit watching putty for four hours for the cursor to reappear ...

 

[MGSteve]

Here are the stats from our 10m+ import.

More info including server specs etc can be found at http://tech.lazyllama.com/2011/08/08/converting-a-big-board-vbulletin-to-xenforo-migration/

What surprised me was that the live import wasn't much faster than the test despite vastly differing specs on the test and live server.

Ah, I was about to post the stats from when I did a test upgrade with beta5 because I noticed your post / thread time was way longer than mine and realised I'd done it with the vB 3.x importer - is the vb 4 really that much slower?

I've done a test run through of mine,

371,568 threads,
4m posts,
55,000 users

I pruned off all the inactive users with no activity & no posts in the last 12 months, pruned off all the banned accounts over 12 months old and removed all the accounts awaiting activation over 12 months old.

So, we pruned off about 1200 posts & 30,000 user accounts (!!) The forum is 8 years old, so a fair amount of junk in there, to be quite honest.

Here's the import screen, so you can see the times. The server in this case was my desktop W7 machine (Core i7 @ 3.6Ghz, 12GB Ram, Raptor HDD), but as others have said, load is pretty minimal during transfer. It was markedly slower (posts alone took 8 hours) the first time through when the Web server was on one server and the DB was my desktop. It may well be worth doing the migration with the web server & DB on the same box and then move the web files off onto the normal server if you have them separate.

The only thing I can't do is rebuild the search indexes at the moment as there is a bug in b5.

The attachments speed was hampered by the files being on the file server, so there was some LAN overhead there. It should be quicker if they're on the same machine.

 

[Kier]

This is great news...

What might be nice Kier is if there was something completely out of browser for the larger boards that can be done from a command line perspective...

It is indeed a command-line system.

 

[Carlos]

Ah, I was about to post the stats from when I did a test upgrade with beta5 because I noticed your post / thread time was way longer than mine and realised I'd done it with the vB 3.x importer - is the vb 4 really that much slower?

Yes, it is.

 

[EasyTarget]

Ok, thanks to everyone, ill contact anyone how offered hes help if he(the owner) decide to use xenf.
right now it looks like the best option, but its alot of work, and we may reconsider converting the forum because of it(and bacause the forum is heavly moded).
again, thanks to everyone!

Just curious with a forum that large...would it be possible to import in phases? Split up all the posts by year? Import one year at a time. So when it came time to make the switch to production, you would only need to import maybe the last months postings?

 

[Paul B]

You can't do incremental imports.

 

[Floris]

cli php execution might be nice though.

 

[Wildcat Media]

We've been working together with some very large boards to produce an importer that can take better advantage of server resources. In some cases it can be several time faster than the standard importer. It still needs work, but it's on the way...

If I can convince our site owner to convert our big board to XF in the near future, you're welcome to use it as the proverbial guinea pig. Or, I'd be willing to help test it. I don't fancy being offline for 24 hours to convert, although it will be well worth it. Anything speeding it up would help.

 

[MGSteve]

Yes, it is.

Another good reason to stay on vB 3.x then!

Can't wait to give this command line version a try

 

[shock]

Just curious with a forum that large...would it be possible to import in phases? Split up all the posts by year? Import one year at a time. So when it came time to make the switch to production, you would only need to import maybe the last months postings?

I dont know i that is possible, but there is now way we gonna import only a part of it, we want it all.

 

[MGSteve]

A proper setup for 1 million posts can take an hour, not including the search rebuild.
Please use the cli to rebuild search index, don't use the xenforo browser option.

Sorry to probably ask an obvious question, where is this CLI (I presume a script) to rebuild them outside of the browser interface?

 

[CTXMedia]

Sorry to probably ask an obvious question, where is this CLI (I presume a script) to rebuild them outside of the browser interface?

I don't think it's been released yet. Kier?

 

[lmq]

Look in the library/XFCliImporter folder. There’s an import.php script for running from the command line and an XF add-on. I don’t exactly know how to use this, but it’s definitely in the 1.1 beta releases.

 

[Deebs]

You need to install the addon, then select the appropriate importer (it has alpha in the title). It works fine as I helped test it. So far it only works under Linux.

Currently doing a test import using this importer and 1.1 b2 for a 4 million post forum.

 

[Deebs]

Using the importer I have imported 50,000 threads in around 30 minutes (cannot tell you the number of posts included in that as InnoDB does not return accurate row counts)