XF 2.0 vB 3.8 -> XF 2.0.1 import and invalid UTF-8 errors

[chrberni]

Attempting to import our vB database, I repeatedly see this, for example:

[InvalidArgumentException]​

Received invalid UTF-8 for string column [custom_title]​

​

[InvalidArgumentException]​

Received invalid UTF-8 for string column​

​

xf:import​

debug_backtrace information:

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Import/Data/EntityEmulator.php : 75 : castValueToType​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Import/Data/User.php : 123 : set​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Import/Data/AbstractData.php : 292 : set​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Import/Importer/vBulletin.php : 1016 : __set​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Import/Importer/vBulletin.php : 929 : setupImportUser​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Import/Runner.php : 160 : stepUsers​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Import/Runner.php : 74 : runStep​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Cli/Command/Import.php : 66 : run​

- /home/chris/TouchArcade/public_html_xenforo/src/vendor/symfony/console/Command/Command.php : 242 : execute​

- /home/chris/TouchArcade/public_html_xenforo/src/vendor/symfony/console/Application.php : 843 : run​

- /home/chris/TouchArcade/public_html_xenforo/src/vendor/symfony/console/Application.php : 194 : doRunCommand​

- /home/chris/TouchArcade/public_html_xenforo/src/vendor/symfony/console/Application.php : 117 : doRun​

- /home/chris/TouchArcade/public_html_xenforo/src/XF/Cli/Runner.php : 63 : run​

- /home/chris/TouchArcade/public_html_xenforo/cmd.php : 15 : run​

​

"custom_title" is just one of the columns this will happen with. Others are: "username", "signature", "about". Searching around, it seems that this is likely to be addressed in version 2.0.2.

Until that release, I am wondering how I might just hack a few lines somewhere to ignore the value (null string instead) and echo out what table, column, and row id caused it. It is just difficult to find a location in the code where vars like $tablename, $columnname, $rowid are available. Can you point me in a direction for that?

 

[convolutional]

I've started testing out an import of a similar forum (vb 3.8). I ran it via the web interface, which is probably a bad idea, but I'm just testing things out. The import died when it got to the posts table. Trying to restart the import on the posts table fails immediately with the error included below, so I guessed it was a problem with the very first batch of posts.

I started looking at what did import into xenforo, and there was a giant block of post ids missing (skipped from 2536 to 9xx,xxx). So I looked at post 2537 in the vbulletin table and... it's a single character, 0x0a, a line feed.

It might not be the sort of printable character you ideally want in a post, but it's perfectly valid utf-8, so I think xf is being too aggressive in what it considers invalid?


InvalidArgumentException: Received invalid UTF-8 for string column [message]

• src/XF/Import/Data/EntityEmulator.php:79
• Generated by: adminusername
• Jan 29, 2018 at 9:24 AM

Stack trace
#0 src/XF/Import/Data/AbstractEmulatedData.php(24): XF\Import\Data\EntityEmulator->set('message', '\n', Array)
#1 src/XF/Import/Data/AbstractData.php(97): XF\Import\Data\AbstractEmulatedData->set('message', '\n', Array)
#2 src/XF/Import/Importer/vBulletin.php(2934): XF\Import\Data\AbstractData->bulkSet(Array)
#3 src/XF/Import/Runner.php(160): XF\Import\Importer\vBulletin->stepPosts(Object(XF\Import\StepState), Array, 8)
#4 src/XF/Import/Runner.php(74): XF\Import\Runner->runStep('posts', Object(XF\Import\StepState), 8)
#5 src/XF/Admin/Controller/Import.php(231): XF\Import\Runner->run()
#6 src/XF/Mvc/Dispatcher.php(249): XF\Admin\Controller\Import->actionRun(Object(XF\Mvc\ParameterBag))
#7 src/XF/Mvc/Dispatcher.php(88): XF\Mvc\Dispatcher->dispatchClass('XF:Import', 'Run', 'html', Object(XF\Mvc\ParameterBag), 'importData', Object(XF\Admin\Controller\Import), NULL)
#8 src/XF/Mvc/Dispatcher.php(41): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#9 src/XF/App.php(1880): XF\Mvc\Dispatcher->run()
#10 src/XF.php(328): XF\App->run()
#11 admin.php(13): XF::runApp('XF\\Admin\\App')
#12 {main}

-------------

Previous InvalidArgumentException: Received invalid UTF-8 for string column - src/XF/Mvc/Entity/ValueFormatter.php:100
#0 src/XF/Import/Data/EntityEmulator.php(75): XF\Mvc\Entity\ValueFormatter->castValueToType('\n', 5, Array)
#1 src/XF/Import/Data/AbstractEmulatedData.php(24): XF\Import\Data\EntityEmulator->set('message', '\n', Array)
#2 src/XF/Import/Data/AbstractData.php(97): XF\Import\Data\AbstractEmulatedData->set('message', '\n', Array)
#3 src/XF/Import/Importer/vBulletin.php(2934): XF\Import\Data\AbstractData->bulkSet(Array)
#4 src/XF/Import/Runner.php(160): XF\Import\Importer\vBulletin->stepPosts(Object(XF\Import\StepState), Array, 8)
#5 src/XF/Import/Runner.php(74): XF\Import\Runner->runStep('posts', Object(XF\Import\StepState), 8)
#6 src/XF/Admin/Controller/Import.php(231): XF\Import\Runner->run()
#7 src/XF/Mvc/Dispatcher.php(249): XF\Admin\Controller\Import->actionRun(Object(XF\Mvc\ParameterBag))
#8 src/XF/Mvc/Dispatcher.php(88): XF\Mvc\Dispatcher->dispatchClass('XF:Import', 'Run', 'html', Object(XF\Mvc\ParameterBag), 'importData', Object(XF\Admin\Controller\Import), NULL)
#9 src/XF/Mvc/Dispatcher.php(41): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#10 src/XF/App.php(1880): XF\Mvc\Dispatcher->run()

 

[Kier]

There are major changes coming in 2.0.2 that resolve all of these UTF8 errors.