Manual import questions

[Merri]

Background

I'm working on a manual import from a very heavily modded phpBB2. This forum has been previously partially imported into MyBB, which is what I have already successfully imported to XenForo in a test run.

Now I'm working on fixing the issue of having two forums running, the phpBB2 one still running in read only mode due to historical value of old threads that people want to retain and check out every now and then. I can't use MyBB import again because I already have new users, threads and posts to deal with. The only things I want to import from phpBB2 are the missing users, threads and posts. Maybe private messages too if I go insane enough.

The Questions

Linking existing information together to avoid duplicate data is easy enough. However with users I have one issue that is a concern: how I should deal with passwords? Does XenForo require password fields to exist for a user or can I just ignore them? Or is it (easily) possible to extend XenForo's password system to automatically upgrade passwords from phpBB2 format like MyBB does?

As a generic import question: can I just add the missing data to XenForo tables and then rebuild caches?

 

[Mike]

You need to have an entry in every table that the XF system inserts users into. This includes the table that holds authentication information. You can have entries based on no password, but you should just make an authentication system to handle your passwords. You can see various examples of this in the importers. (I recommend you write your code to insert using the standard importer tools if possible.)

 

[Merri]

Thanks. I took the phpBB3 importer and I'm now simplifying it for my needs.

In the meanwhile I think I found a bug in the phpBB3 importer:

        if ($user['group_id'] == 3) // coppa
        {
            $import['user_state'] = 'moderated';
        }
        else if ($user['user_type'] == 1 && $user['user_inactive_reason'] == 1) // inactive at registration
        {
            $input['user_state'] = ($this->_userActivationSetting == 2 ? 'moderated' : 'email_confirm');
        }
        else if ($user['user_type'] == 1 && $user['user_inactive_reason'] == 2) // inactive at profile edit
        {
            $input['user_state'] = 'email_confirm_edit';
        }
        else
        {
            $import['user_state'] = 'valid';
        }

The two rows starting with $input should probably be $import?

 

[Mike]

Indeed, good catch there. I've fixed that for 1.4.3.

 

[Merri]

To update the status: I'm now successfully live with nearly 240 000 posts! Creating my own phpBB2 importer for the specific needs I had was simple enough. Just a lot of attention to detail and fixing a couple of crashing typos and I had a successful full merge. In total I needed five full test runs to fix issues and be confident enough that I could go live with the next full run. For the most part the XF's importer works very well with the biggest issues that there are.

Only limitation that was kind of meh was that I couldn't give a specific user ID if I already knew that some user exists in XenForo's user table. The user's posts simply appeared as ones made by a guest. Instead everything had to go via merge, so thus I updated the old phpBB2 user table to contain fully matching email and username in the cases where I already knew who was who. The end result worked as expected.

Other things that were issues were colliding nicknames, which had some severe historical issues where UTF-8 conversion had gone wrong, and thus two users could have a nearly identical nickname that wouldn't have been allowed if UTF-8 was properly stored. In addition there were new users with same nicknames that existed in the old phpBB2 but not in MyBB. This lead to me selectively renaming users either in source MyBB table before conversion or during conversion depending on which user seemed more deserving of keeping their nickname. MyBB's importer was evil in that it's user merger seemed to most often ignore both users with similar but not fully matching nicknames...

Don't know if there is much more to tell on this experience. It was far less painful than any previous import I've had to do.