Questions about the MyBB Importer

[Vekseid]

I am looking to move my smaller forum, a MyBB installation (~2 million posts, ~20 million private messages) over to XenForo 2 sometime after its release.

- Does the importer copy over the 'built in' MyBB profile fields like the Personal Notepad into XenForo's own field system?

- Does the importer do a live-running import for PMs and posts? Especially for PMs, I would like to have my forum up and running ASAP rather than wait however long it takes the importer to walk over 20 million records.

- If not, is the code for this fairly transparent so I can make it so?

- Regarding PMs, anyone familiar with MyBB's atrocious schema can imagine the situation my forum is in with 20 million pms. How good is the importer at handling discrepencies? Restoring lost/deleted pms, and so on?

- How does the importer handle custom BBCode, or how should I handle this in general?

Thank you for your time.

 

[Paul B]

Custom profile fields should be preserved.

The source database will have to be locked during the import process to ensure data integrity.
The built in importer doesn't support incremental/top up imports so the site would be down for however long it takes to import the content.

Custom bb codes won't be imported - you will just need to create them in the ACP to match what you already have.

 

[Vekseid]

Custom profile fields should be preserved.

This is not an answer to my question. MyBB has some hardcoded fields - my question was does the importer handle these?

The source database will have to be locked during the import process to ensure data integrity.
The built in importer doesn't support incremental/top up imports so the site would be down for however long it takes to import the content.

This also doesn't answer my question.

For a live import, I mean setting the auto increment starts to the last record + 1 as needed, and importing the data as the new site runs. Does the MyBB importer do this?

 

[TPerry]

If you are asking if the new site can be live when the import of the myBB is occurring.. if I remember correctly the new site needs to be disabled (XenForo) so that there is no user access.

If you are asking if you can have data in place on the XenForo site and THEN import myBB into it - last time I did an import that was not able to be done. You import into a virgin database with no other content (user based) than the default SuperAdmin user.

 

[Mike]

(Note that you mention XF2, but the comments here are about the XF1 MyBB importer. Concepts may differ when the XF2 MyBB importer becomes available, but until then, you'll need to import on 1.5 and then upgrade.)

I don't recall the MyBB schema well at this point, though we do import into our "about" field a value from "fid2" and "fid1" into our location field. Based on what I have "fid2" is a custom field called "Bio". I'm not sure whether this is the "Personal Notepad" you refer to.

I'm not really totally following what you mean by the "live running" import, though if you mean the new site, that really does need to be closed while the import is executing. Various things won't happen until the end of the import.

 

[Vekseid]

The personal notepad is a user-only visible field that they can edit for whatever personal reasons they wish. It is the 'notepad' column in the users table. If importing into SMF/Elkarte for example I would make this into a text field only the user + admin could see and edit.

There is also a 'usernotes' column which is for moderator notes on specific members.

fid1 and fid2 are the default custom fields and are 'Location' and 'Bio' respectively.

And that is rather unfortunate. If not fixed in the XF2 importer, one way or another I would have to splice out at least the PM import code, make sure that the private conversation auto-increment ids are set to a comfortably high enough value to cover for importing live, and either write my own script for them, or modify yours to suit the task.

 

[Suzanne O]

The personal notepad is a user-only visible field that they can edit for whatever personal reasons they wish. It is the 'notepad' column in the users table. If importing into SMF/Elkarte for example I would make this into a text field only the user + admin could see and edit.

There is also a 'usernotes' column which is for moderator notes on specific members.

fid1 and fid2 are the default custom fields and are 'Location' and 'Bio' respectively.

And that is rather unfortunate. If not fixed in the XF2 importer, one way or another I would have to splice out at least the PM import code, make sure that the private conversation auto-increment ids are set to a comfortably high enough value to cover for importing live, and either write my own script for them, or modify yours to suit the task.

You need to change the settings of the user so they don't see what you see as the admin.

 

[TPerry]

And that is rather unfortunate. If not fixed in the XF2 importer,

Not really anything to "fix" as those fields do not have corresponding default equivalents in the standard XenForo install if I remember correctly. You would have to create custom fields for that and after doing so would require customization of the importer (or a custom script to pull in that data after initial import) to map to those admin created custom fields.

Expecting to import one scripts extra fields into another script that does not use them is actually somewhat unrealistic.

And I don't really understand the fixation on "importing live" as I am not aware of any commercially available script that will allow you to keep the old site online and active and be importing to the new script and automatically update any information/users/posts that occur during the importation (and I've used IPS, WBB and XenForo).

 

[Vekseid]

Not really anything to "fix" as those fields do not have corresponding default equivalents in the standard XenForo install if I remember correctly. You would have to create custom fields for that and after doing so would require customization of the importer (or a custom script to pull in that data after initial import) to map to those admin created custom fields.

I was referring to the lack of live importing in my response to @Mike there, not the fields.

Expecting to import one scripts extra fields into another script that does not use them is actually somewhat unrealistic.

Except, yanno, it does.

And I don't really understand the fixation on "importing live" as I am not aware of any commercially available script that will allow you to keep the old site online and active and be importing to the new script and automatically update any information/users/posts that occur during the importation (and I've used IPS, WBB and XenForo).

The dev at least understands my question, but to be clear, I am referring to having the new site running and accepting posts and pms while I import the largest table from the old.

If you do not understand why I am rather tepid about shutting down my site while it imports 20 million records through a web script...

 

[TPerry]

The dev at least understands my question, but to be clear, I am referring to having the new site running and accepting posts and pms while I import the largest table from the old.

Oh, I'm aware of what you meant... and I'm aware that IPS and WBB also do not do this. You are expected to have both sites closed at time of importation and THEN open the new site.

Except, yanno, it does.

Except.. yanno... not really as it does not have a corresponding field to place them into. Do you expect their import to create a custom field for you to use based upon what your old DB has? Sounds like you need to hire out a developer to create a custom importer - or write one for yourself that performs to your expectations.

If you do not understand why I am rather tepid about shutting down my site while it imports 20 million records through a web script...

That is a desire of yours that none of the 3 scripts I mentioned will provide AFAIK.

Pretty sure each of those 3 require that the site be shut down... and there have been many sites of much larger post count that have had to go through the same thing. As long as you have adequate hardware, the importation should not be excessive.

One example from a site I participate at: