XF 1.3 Error importing from IPB3.4 to XenForo

[Chromaniac]

I am testing migration from IPB to XenForo. I am facing issues in the first import step itself:

Mysqli statement execute error : Duplicate entry '3' for key 'PRIMARY'

    Zend_Db_Statement_Mysqli->_execute() in Zend/Db/Statement.php at line 297
    Zend_Db_Statement->execute() in Zend/Db/Adapter/Abstract.php at line 479
    Zend_Db_Adapter_Abstract->query() in Zend/Db/Adapter/Abstract.php at line 574
    Zend_Db_Adapter_Abstract->insert() in XenForo/DataWriter.php at line 1624
    XenForo_DataWriter->_insert() in XenForo/DataWriter.php at line 1613
    XenForo_DataWriter->_save() in XenForo/DataWriter.php at line 1405
    XenForo_DataWriter->save() in XenForo/Model/Import.php at line 599
    XenForo_Model_Import->_importData() in XenForo/Model/Import.php at line 637
    XenForo_Model_Import->importUserGroup() in XenForo/Importer/IPBoard.php at line 352
    XenForo_Importer_IPBoard->stepUserGroups() in XenForo/Importer/Abstract.php at line 97
    XenForo_Importer_Abstract->runStep() in XenForo/ControllerAdmin/Import.php at line 187
    XenForo_ControllerAdmin_Import->_runStep() in XenForo/ControllerAdmin/Import.php at line 239
    XenForo_ControllerAdmin_Import->_startStep() in XenForo/ControllerAdmin/Import.php at line 182
    XenForo_ControllerAdmin_Import->actionStartStep() in XenForo/FrontController.php at line 347
    XenForo_FrontController->dispatch() in XenForo/FrontController.php at line 134
    XenForo_FrontController->run() in /home/test/domains/beta.test.in/public_html/admin.php at line 13

It is a fresh install with default data. What should I do?

 

[Mike]

If you're maintaining IDs, there's a bit of a limitation when it comes to group mapping in the import -- you have to map groups 1 through 4. Those are standard groups in XF that can't be removed. If you have a group with an ID of 1 to 4 that isn't mapped to a standard XF group, it will fail to create like this.

The alternative is to not maintain IDs.

 

[Chromaniac]

Alright. I got around that issue. A lot of content got imported. I got stuck importing at polls. And it would not let me access the import page at all now.

Here is the error.

preg_match() expects parameter 2 to be string, array given

    XenForo_Application::handlePhpError()
    preg_match() in XenForo/Importer/Abstract.php at line 177
    XenForo_Importer_Abstract->_convertToUtf8()
    array_map() in XenForo/Importer/IPBoard.php at line 2970
    XenForo_Importer_IPBoard->stepPolls() in XenForo/Importer/Abstract.php at line 97
    XenForo_Importer_Abstract->runStep() in XenForo/ControllerAdmin/Import.php at line 187
    XenForo_ControllerAdmin_Import->_runStep() in XenForo/ControllerAdmin/Import.php at line 135
    XenForo_ControllerAdmin_Import->actionImport() in XenForo/FrontController.php at line 347
    XenForo_FrontController->dispatch() in XenForo/FrontController.php at line 134
    XenForo_FrontController->run() in /home/test/domains/beta.test.in/public_html/admin.php at line 13

 

[Chromaniac]

Ok. I managed to get around that from information found in another thread on the forum. I have setup the redirection scripts as instructed. Since my forum is at a test location, I am not sure if it works fine...

Original URL:
broadbandforum.co/topic/55143-transformers-age-of-extinction/
New URL:
beta.broadbandforum.in/threads/transformers-age-of-extinction.55143/

I tried loading
beta.broadbandforum.in/topic/55143-transformers-age-of-extinction/

It redirected to the homepage of the forum. Is this normal? Or I messed up something?

Update: I forgot to uncomment the config line. Appears to be working fine now. Cool.

 

[Chromaniac]

I have another issue which is one of the several reasons I am considering dumping IPB for XF. Few days ago, IPB guys posted about the upcoming launch of IPB4. Exciting considering it was supposed to come out last year. Anyways. One of the things they suggested we do to prepare for its launch was to convert the database to UTF8. I did exactly that only to find out (a few weeks later) that it does not support some character types like emoji. I renewed my license. Asked for support only to be told that I would have to revert back to the older database and convert it to UTF8MB4 something.

My XF test install running the imported database has the exact same issue. I guess that was inevitable. I want to know how I can get it changed to UTF8MB4. I am completely clueless when it comes to SQL and Databases and Encoding stuff. Any suggestions would be appreciated. Thanks.

 

[Mike]

In terms of the poll issue, I'd need access to the import (and files) to debug it. The poll appears to be in a format that isn't expected. (If you want to provide this, please submit a ticket.)

In terms of UTF8MB4, it's a different DB charset. All the tables/columns would need to be changed and it may have other knock on effects. Things like this may need to be applied on upgrades as well. As such, it's not really something that we can officially support.

 

[Chromaniac]

You mean to say you cannot support a database in UTF8MB4 charset or you cannot support the conversion process? What charset should I try to get my database into for XenForo? IPB4 is UTF8 only which is why they released the script to convert the database. Big mistake following their advice before the release of IPB4. Thanks.

 

[Mike]

We use the UTF8 character set in MySQL. UTF8MB4 is a different (extended) version. It's not something we use or test against (it affects things like index lengths) and the code is based on MySQL's UTF-8 support (max 3 byte characters, just the BMP).

So both -- we can't provide official support for a DB using UTF8MB4 as you might run into issues or it might not be a panacea for your issues since the code is not built for it.

In terms of the import, the importer will handle the conversion to UTF-8 from whatever charset you're using (as listed in your IPB options).

 

[Chromaniac]

Ook... So that means I cannot post emoji here as well?

Testing: Get Emoji - Copy and Paste any Emoji

This was unexpected. Thanks for the response though.

 

[Chromaniac]

Alright. Another question. I see XenForo has no inbuilt support for tags. There are addons which is fine. But this means that I would lose all my tags during the import process... Are there any plans to add native support for tags for threads in the future? I can wait for the next major update. Would give me a chance to look at IPB4 as well. Thanks.

 

[Mike]

New features would be announced in the HYS forum.

 

[Chromaniac]

Another question… Are there attributes that I can use to reach the:

Last post in the thread
Newest post in the thread that I have not read

In IPB, I can get ?view=getlastpost and ?view=getnewpost to the end of the URL for these actions. During my brief testing with XenForo I did find out that adding /reply to the thread URL is a good way to get to the reply composer immediately. This is pretty helpful.

Some members like to add bookmarks to their favorite threads in their browser so they like these attributes on my current installation. Any help would be appreciated. Thanks!

 

[Paul B]

/unread will take you to the first unread post.

 

[Chromaniac]

Awesome. Thanks!

 

[Chromaniac]

Just noticed a major issue with the final import I did. All the images are gone. They were imported just fine on my test install. These are embedded images from services like Imgur. Makes no sense. The post content has no bbcode for images. Pretty confusing.

broadbandforum.co/threads/game-of-thrones.56802/ (final install)
beta.broadbandforum.in/threads/game-of-thrones.56802/ (test install)

Any ideas where I messed up? This was part of the post content and post content was imported just fine.

 

[Mike]

Does it appear when you edit? (Try turning off the WYSIWYG editor.)

If not, then it's really not there. I know you used some queries to update the post content so I don't know if there was one that removed the data. If the content is truly not there, you'd need to do a fresh import to restore it.

 

[Chromaniac]

Yup. Not there. They are gone. Made no sense! I did not import the user avatars during import. But I doubt that is the reason.

Anyways, it's manageable. I ran the YouTube queries much later (around 20 minutes ago).

PS. Someone fix the title of this thread. It sounds lame as I wrote important instead of importing. Thanks!

 

[Chromaniac]

Just realized... My test import from IPB was from the database in UTF8 format. The final import was from a database whose encoding had been changed to UTF8MB4. I wonder if this caused the loss of images. Not going to think much about it now. Posting it here in case you might find this information useful. Thanks!

 

[Mike]

That should be unrelated, I would think.