XF 2.2 How long should a vbulletin import take?

[bburton]

We are importing from [ vbulletin version 3.8.4 Patch Level 2 ] --to--> [ XenForo 2.2.10-Patch-1 ] using the importer found in the customer area.

I realize there are many factors to consider, but -very roughly- how many hours would you expect an import of approximately 4 million posts, 80k attachments (all in the database) and 20GB of data would take?

How much faster is the command line import method compared to the browser method? Twice as fast? Ten times faster?

What can slow down the import, and what can speed it up?

Thanks!

 

[briansol]

server speed will dictate it a lot.

for sure, use the CLI interface. it's 10x at least and more reliable.

from the hip, expect bout an hour per million, heavily padded. so your 4 mil will probably take 2-3 hours, again, depending on your hardware.

It's little things that add up too -- like lots of PMs, prodile bits, groups and settings, etc.

You might want to consider moving the attachments out of the db into the filesystem on vb first. xf doesn't store attachments in the db (not sure how the conversion process works)

 

[AndyB]

how many hours would you expect an import of approximately 4 million posts

With that size forum, taking at least 24 hours would be normal.

 

[djbaxter]

On one large forum I manage it took 8-10 hours (from vB 4.25).

 

[Slavik]

With that size forum, taking at least 24 hours would be normal.

What?

If a 4 mil post forum takes more than a few hours there something wrong.

On any half decent hardware with CLI I would probably expect this to be <2 hours tbh with the majority of that time spent on the rebuilds.

 

[djbaxter]

What?

If a 4 mil post forum takes more than a few hours there something wrong.

On any half decent hardware with CLI I would probably expect this to be <2 hours tbh with the majority of that time spent on the rebuilds.

As I said above, although I've never seen it take 24 hours, using CLI I have seen it take 8-10 hours with medium large to large forums (that's on a dedicated server). It does of course also change with the speed of your server and generally what I do is set PHP to unlimited for purposes of the import (you can change it back after the import).

 

[Slavik]

As I said above, although I've never seen it take 24 hours, using CLI I have seen it take 8-10 hours with medium large to large forums (that's on a dedicated server). It does of course also change with the speed of your server and generally what I do is set PHP to unlimited for purposes of the import (you can change it back after the import).

Define medium to large.

I would be expecting 8-10 hour imports to be limited to boards of 25M+ posts. Or ones with an unusually large quantity of attachments into the hundreds of GB.

 

[briansol]

if you're including rebuilding the search index, i can see it taking a day. but the import itself should only be a few hours.

 

[AndyB]

On any half decent hardware with CLI I would probably expect this to be <2 hours tbh with the majority of that time spent on the rebuilds.

The OP is asking how long it will take not using the CLI, hence why we're indicating it will take many hours and to be patient.

 

[Slavik]

The OP is asking how long it will take not using the CLI, hence why we're indicating it will take many hours and to be patient.

Sorry Andy, claiming 24 hours even on the web importer has absolutely no basis in reality.

 

[AndyB]

Sorry Andy, claiming 24 hours even on the web importer has absolutely no basis in reality.

When I imported my forum from vB4 a number of years ago, it took about 12 hours and that was a 2 million post forum. So I estimated a 4 million post forum could take twice as long, hence why I wrote it could take 24 hours.

 

[Slavik]

When I imported my forum from vB4 a number of years ago, it took about 12 hours and that was a 2 million post forum. So I estimated a 4 million post forum could take twice as long, hence why I wrote it could take 24 hours.

Are you making your assumptions based off the XF1 importers then?

 

[AndyB]

Are you making your assumptions based off the XF1 importers then?

Yes that and over the years hearing how long it takes on average.

 

[Slavik]

Yes that and over the years hearing how long it takes on average.

I think then you need to get some time spend using the XF2 importers and the improvements they have before giving out old information.

 

[Baby Community]

friends, I have switched from mybb to xenforo before. I also migrated it from vbullet to xenforo. I tried to switch from phbb to xenforo. But I have not done it with such a huge database. And I have always done these migrations from the xf1 importer. My question is what is the CLI? @Slavik @AndyB @djbaxter @briansol

 

[djbaxter]

Define medium to large.

I would be expecting 8-10 hour imports to be limited to boards of 25M+ posts. Or ones with an unusually large quantity of attachments into the hundreds of GB.

For two forums that come to mind that I've converted, one was about 89000 posts and the other was 10, 000, 000 posts.

friends, I have switched from mybb to xenforo before. I also migrated it from vbullet to xenforo. I tried to switch from phbb to xenforo. But I have not done it with such a huge database. And I have always done these migrations from the xf1 importer. My question is what is the CLI? @Slavik @AndyB @djbaxter @briansol

It's an abbreviation for Command Line Interface.

This is where you use the Terminal app on your server to execute various commands. It works a lot faster for larger forums than updating from your browser.