vB as an import intermediate, share your experiences

[yoghurtfarmer]

I thought I'd give an overview of some issues I ran into importing from SMF -> vB -> xF. While some of these solutions are SMF specific, they can be easily modified to work for other forum software.

1. Getting passwords to work - DO THIS BEFORE XF IMPORT

A crude fix until someone puts together something better. Remember to expand the password field in the vBulletin database, I wasted a lot of time when I forgot to do this.

http://xenforo.com/community/thread...ing-vbulletin-as-an-import-intermediate.5172/

2. URL redirection
This if for smaller forums and for people without a lot of PHP knowledge but want to do it semi-automatically. No doubt there will be much better solutions down the line.

http://xenforo.com/community/threads/crude-way-of-doing-301-redirects-semi-manually.5842/

3. xenForo will not import threads without postusername set.
So after running Impex but prior to importing into xF. Run the query below in phpmyadmin.

UPDATE `set this to your vB database`.`thread` SET `postusername` = 'Guest' WHERE `thread`.`postusername` =''

5. With SMF, the title of REPLIES will show up bolded at the top of every post in Xenforo
Since most people usually don't edit the title when they reply, this creates clutter.

To remove them, use the following query BEFORE importing into XF.

UPDATE `set this to your vB database`.`post` SET 'title' = ''

4. Broken quotes can be fixed with some additions to impex's cleaner.php

Fortunately someone has taken care of SMF 2 to vB quote formatting already and shared it on vB.com.

http://www.vbulletin.com/forum/showthread.php?309519-SMF-2-Quote-Cleaner-Code

After converting your quotes to vB style and importing them to xenForo, use brendanc's tool to convert the vB style quotes to xF while also fixing the post references.

6. Things to be fixed after importing into xF

Birthday, category descriptions and timezones need to be fixed after import into xF (at least with SMF).

These are the queries that I used to clean them up:

UPDATE `your xenforo database`.`xf_user` SET `timezone` = 'America/Los_Angeles' WHERE `xf_user`.`timezone` ='Europe/London'
UPDATE `your xenforo database`.`xf_node` SET `description` = '' WHERE `xf_node`.`description` ='!Forum description'
UPDATE `your xenforo database`.`xf_user_profile` SET `dob_day` = '0' WHERE `xf_user_profile`.`dob_year` ='1'
UPDATE `your xenforo database`.`xf_user_profile` SET `dob_month` = '0' WHERE `xf_user_profile`.`dob_year` ='1'
UPDATE `your xenforo database`.`xf_user_profile` SET `dob_year` = '0' WHERE `xf_user_profile`.`dob_year` ='1'

"Find all thread started does not" also doesn't seem to work even after rebuilding the search index (see this bug report).

 

[yoghurtfarmer]

For any SMF users, here's what I replaced with cleaner.php (aside from the quote fix), most of them are BB Code and smiley related. The pt] replacement was to change stuff like [size=10pt] to [size=10].

            "[nobbc]"    => "[plain]",
            "[/nobbc]"    => "[/plain]",
            """    => '"',
            ">"    => ">",
            "&lt;"    => "<",
            "&amp;"    => "&",
            ";D"    => ":D",
            "8)"    => ":cool:",
            " ???"    => " :confused:",
            "::)"    => ":rolleyes:",
            ":-["    => ":oops:",
            ":-X"    => "",
            ":-\\"    => "=/",
            ":-*"    => "",
            ":'("    => ":(",
            "[li]"    => "[*]",
            "[/li]"    => "",
            "[hr]"    => "",
            "pt]"    => "]"

 

[glorify]

Totally off topic, but you have done a great job with the import and design of your Xen forum.

 

[lazer]

Totally off topic, but you have done a great job with the import and design of your Xen forum.

Agree with that - great job on the move and tweak of the standard design.

 

[Kim]

Also offtopic... Really nice job, your site looks fab!

 

[steven s]

Very nice. I like the RSS feed to bring in content.

 

[yoghurtfarmer]

Thanks everybody.

 

[Wildcat Media]

This guide will be helpful, as I'm running some test imports for SMF -> vB -> XF.

One thing I've noticed is that avatars do not transfer over from SMF to vB. I have them stored in the attachment directory, so maybe that is why they don't show up...? All directories are readable and writeable, so that is not the issue.

One other thing that has me stumped is password handling:

I thought I'd give an overview of some issues I ran into importing from SMF -> vB -> xF. While some of these solutions are SMF specific, they can be easily modified to work for other forum software.

1. Getting passwords to work - DO THIS BEFORE XF IMPORT

A crude fix until someone puts together something better. Remember to expand the password field in the vBulletin database, I wasted a lot of time when I forgot to do this.

http://xenforo.com/community/thread...ing-vbulletin-as-an-import-intermediate.5172/

I went ahead and made the password field hold 64 characters rather than 32. SMF holds 40 characters. And yet when I do the import, the hashes not only don't match, they are still only 32 characters long. This line in the SMF to vB converter (file 004.php in ImpEx) tells me this:

    $try->set_value('nonmandatory', 'password',                $data['passwd']); // Can't get it, so use this for random

It makes me wonder if ImpEx is just generating random garbage, in the thought that users will need to regenerate their own passwords.

And, there is also Kier's comment regarding usage of other encryption techniques (from the FAQ section):

Q: If I import from [insert forum solution name here], will my members need to reset their passwords?


A: No. XenForo is able to use multiple password and authentication schemes and these can be defined on a per-user basis. If you import from a system that uses md5(md5(password) . salt) as its password hashing scheme, XenForo will allow that user to authenticate using that scheme.

This makes me wonder if modifying XF is unnecessary, or if this is an unimplemented (future) feature.

I don't mind having users redo their avatars and passwords, but as an admin with a conscience, I would rather have these operational as a courtesy.

 

[Dean]

I don't believe Kier was referring to using vb as an intermediate pass through medium. I believe he was referring to ___ -> xf converters that they will write.

Last I checked, and I do not think it has changed, Impex smf -> vb will loose passwords. Note: I did not read the thread in post Number 1 about this.

 

[Wildcat Media]

Hi Dean,

That first link, if I understand it correctly, advises a series of modification to do two things: 1) preserve the password hash from SMF to vB (by way of making the password field larger), then transfering that to XF intact. Then, 2) modifying XF to use the encryption style that SMF uses. I don't want to do that, as each time XF is updated, you're having to modify the script all over again.

Two of my staff have just told me to forget about the avatars (not a big deal), and the passwords (an inconvenience, but saves me a lot of work).

 

[Andy.N]

Quick question
If I have an existing XF forum and I want to merge another existing SMF forum on another domain, is it possible?
It does seem to require a builtin or some add on that provide SMF support.

 

[Dean]

Hi Dean,

That first link, if I understand it correctly, advises a series of modification to do two things: 1) preserve the password hash from SMF to vB (by way of making the password field larger), then transfering that to XF intact. Then, 2) modifying XF to use the encryption style that SMF uses. I don't want to do that, as each time XF is updated, you're having to modify the script all over again.

Two of my staff have just told me to forget about the avatars (not a big deal), and the passwords (an inconvenience, but saves me a lot of work).

Just wow. I am far too lazy to ever try to maintain a system like that...

I would so a simple 2 step import and have users reset their passwords and avatars. At least on a test site to see what is involved.

You could always use 'embarrassing' or 'undesirable' default avatars to urge people to have unique avatars sooner rather than later...

 

[Wildcat Media]

Quick question
If I have an existing XF forum and I want to merge another existing SMF forum on another domain, is it possible?
It does seem to require a builtin or some add on that provide SMF support.

I'm not sure, but I think XF does what I call "incremental" imports. IOW, some importers will completely wipe a forum's tables clean and import, but I believe XF adds imports to existing content. I have thought of merging a couple of forums into our new one but it would probably not go over very well.

 

[yoghurtfarmer]

It's been awhile but since I had no clue what each impex file was doing, I basically just edited any file containing the word password that seemed to be relevant.

ImpExDatabase.php
ImpExDatabase_309.php
ImpExDatabase_350.php

For those files, find this section:

                $newpassword = '';
                $salt =    $this->fetch_user_salt();

Below that should be a snippet of code that's close to what I have below. Replace it with what I have to prevent rehashing.

        if ($this->_password_md5_already)
                 {
                      $newpassword = $this->get_value('nonmandatory', 'password');
                 }
                 else
                 {
                     $newpassword = $this->get_value('nonmandatory', 'password');
                 }

Once you've tweeked impex to maintain the original hash, you still need to make an edit to the xenforo's vBulletin importer to import the username as the salt rather than vB's salt. This part is detailed in the other thread.

As for vB authentication, it's one file that you have to reupload with each update. I think it should also be possible to create a custom authentication scheme by duplicating /library/XenForo/Authentication/vBulletin.php, editing that one line and then renaming it. Then in the database, you would change scheme_class in xf_user_authenticate to match the new authentication scheme. I haven't tested it but if it works you don't have to worry about your edits being overwritten during updates.

 

[Wildcat Media]

Thanks for the tips!

BTW, I wish there was a way to compile all of these conversion tips into one place. The problem I have is that most of us refer to it as "SMF," which we can't search on (being only three letters). I only found this after spending a couple of hours browsing through threads here.

 

[Wildcat Media]

So far, I am finding out the following regarding converting from SMF2 --> vB3.8 --> XF1:

1) User avatars do not import;
2) Post attachments do not import;
3) Passwords will not transfer;
4) Quote BBCode is broken, and the cleaner.php tool in ImpEx does not let me know if it should be run after converting from SMF2 to vB, but before converting vB to XF. I just ran cleaner.php on my vB database, then imported to XF, and the quotes are still broken. (Although running cleaner.php does show the results.)

I wish I could get past these hurdles, but may have to live with a sloppy import.

 

[yoghurtfarmer]

I got past 3 and 4... How come the passwords don't transfer for you?

What do the broken quotes look like?

 

[Wildcat Media]

I got past 3 and 4... How come the passwords don't transfer for you?

What do the broken quotes look like?

Passwords are encrypted differently and won't work. I know there are workarounds but I don't have time to monkey around with everything.

The vB-style quotes are in a different style than those from SMF. This post at The Other Place™ tells what cleaner.php does and what it fixes:

http://www.vbulletin.com/forum/showthread.php/309519-SMF-2-Quote-Cleaner-Code

It's really irritating me that the importer can't do the attachments or the avatars. The importer script is looking for the original filename, but the files are actually stored using some kind of encoding or hash (?) in the actual attachments directory. So, to me ImpEx is broken in that way. Avatars are not a huge deal, but some of the attachments are going to take awhile for me to save and re-attach to the posts. Yes, some are that important, and I can't ask visitors to do this since it's not their fault the attachments aren't converting.

 

[Paul B]

Sadly I seem to have lost all of my attachments too going phpBB -> vB -> XF

And BB Code is a disaster.

At least all the main content and members are OK.

 

[Wildcat Media]

The cleaner.php script is configurable to take care of replacing SMF's BBCode style with that used by either vB or XF. But, I won't know exactly what is affected until I do another import. On our SMF board, we pretty much left the BBCode alone, so most of it is plain vanilla. The only add-on I made was for a YouTube BBCode, but with XF linking media automatically now, there's no need for it.

The odd thing is that I ran cleaner.php, actually saw the changes being displayed while it was run (and the changes were correct), redid the import, but the changes apparently were not written to the database...? I'm not sure just yet. The cleaner.php script does the proper substitutions, but I need to see if it's a problem with the script or something I messed up myself. (Since it uses the ImpEx config file, I don't see how I could be writing to a wrong database.)

More later. And, I will take some notes so that we have something to go back on.

Searching the vB site did me no good either...help seems to be lacking over there lately.

(BTW, it may be more helpful if we had a separate Conversion category here, with one subforum for each system...we can't search for "SMF", and it would be nice to have these categorized all in one place.)