XF 2.3 cli xf:import bug in 8.2/8.3.14 - how to import?

[frm]

It can't work on 8.2:

PHP Fatal error:  Uncaught Error: Call to undefined function mysqli_report() in /var/www/vhosts/.com/httpdocs/src/XF.php:160
Stack trace:
#0 /var/www/vhosts/.com/httpdocs/src/XF.php(112): XF::standardizeEnvironment()
#1 /var/www/vhosts/.com/httpdocs/cmd.php(14): XF::start('...')
#2 {main}
  thrown in /var/www/vhosts/.com/httpdocs/src/XF.php on line 160

And it can't work on 8.3.14 erroring out with another error (something to do with mbstring).

I've only found something on here relating to the importers running correctly on PHP 8.3.13.

Resulting in trying to downgrade PHP to try, or the web.

But, the web produces this:

XF\Db\Exception: MySQL query error [1114]: The table 'xf_post' is full in src/XF/Db/AbstractStatement.php at line 230
XF\Db\AbstractStatement->getException() in src/XF/Db/Mysqli/Statement.php at line 207
XF\Db\Mysqli\Statement->getException() in src/XF/Db/Mysqli/Statement.php at line 83
XF\Db\Mysqli\Statement->execute() in src/XF/Db/AbstractAdapter.php at line 96
XF\Db\AbstractAdapter->query() in src/XF/Db/AbstractAdapter.php at line 219
XF\Db\AbstractAdapter->insert() in src/XF/Import/Data/EntityEmulator.php at line 336
XF\Import\Data\EntityEmulator->insert() in src/XF/Import/Data/AbstractEmulatedData.php at line 73
XF\Import\Data\AbstractEmulatedData->write() in src/XF/Import/Data/AbstractData.php at line 132
XF\Import\Data\AbstractData->save() in src/addons/XFI/Import/Importer/IpsForums.php at line 1789
XFI\Import\Importer\IpsForums->stepPosts() in src/XF/Import/Runner.php at line 244
XF\Import\Runner->runStep() in src/XF/Import/Runner.php at line 61
XF\Import\Runner->run() in src/XF/Admin/Controller/ImportController.php at line 247
XF\Admin\Controller\ImportController->actionRun() in src/XF/Mvc/Dispatcher.php at line 362
XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 264
XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.php at line 121
XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 63
XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2826
XF\App->run() in src/XF.php at line 806
XF::runApp() in admin.php at line 15

 

[frm]

Built 8.3.13

Fatal error: Uncaught Error: Call to undefined function mysqli_report() in /var/www/vhosts/.com/httpdocs/src/XF.php:160
Stack trace:
#0 /var/www/vhosts/.com/httpdocs/src/XF.php(112): XF::standardizeEnvironment()
#1 /var/www/vhosts/.com/httpdocs/cmd.php(14): XF::start('/var/www/vhosts...')
#2 {main}
  thrown in /var/www/vhosts/.com/httpdocs/src/XF.php on line 160

 

[frm]

Trying

ALTER TABLE xf_post MAX_ROWS=1000000000;

As a hail mary before going to bed.

 

[TMC]

Built 8.3.13

Fatal error: Uncaught Error: Call to undefined function mysqli_report() in /var/www/vhosts/.com/httpdocs/src/XF.php:160
Stack trace:
#0 /var/www/vhosts/.com/httpdocs/src/XF.php(112): XF::standardizeEnvironment()
#1 /var/www/vhosts/.com/httpdocs/cmd.php(14): XF::start('/var/www/vhosts...')
#2 {main}
  thrown in /var/www/vhosts/.com/httpdocs/src/XF.php on line 160

Usually these mean that mysqli support/extension isn't enabled on your PHP version. Have you checked that it's enabled?

 

[frm]

Usually these mean that mysqli support/extension isn't enabled on your PHP version. Have you checked that it's enabled?

Will check after this web attempt. Restarted mysql with

max_heap_table_size = 12884898816
innodb_data_file_path = ibdata1:10M:autoextend

 

[frm]

Trying

ALTER TABLE xf_post MAX_ROWS=1000000000;

max_heap_table_size = 12884898816
innodb_data_file_path = ibdata1:10M:autoextend

One of these got it past 60% failure to import 1.65M posts (100%); I don't know which one. But, from my limited understanding, innodb_data_file_path = ibdata1:10M:autoextend may make the database expand to a point of not being able to access the server.

Also, it took an hour... whereas xf:import is supposedly quicker (web is 1000 rows per second, thereabouts).

This was a test run (from the morning) to try and get done anyway, so I will probably check up on

mysqli support/extension isn't enabled on your PHP version

This to see if I can run it much quicker and do multiple test imports in a lot less time.

Still on step 13/15, so something may still go wrong. But, the bulk of the data appears to have been successfully imported.

I hope xf:import with mysqli support (if that's the case) will work quicker than this, cause I probably have to do it 2 or 3 more times before moving to production...

 

[Chris D]

Sometimes the PHP binary available on the CLI is not the same as the one used by the web server so that’s where I would start.

which php will tell you what php is linked to. php -v will tell you the version which you may find is different to what you’re running via the web.

If the error continues or the output of the above looks to be correct you may need to look into why mysqli_report is not available.

 

[frm]

Sometimes the PHP binary available on the CLI is not the same as the one used by the web server so that’s where I would start.

I noticed that with it being 8.1 while the server was 8.3 (then 8.2), but still didn't understand why it'd throw the errors. I figured the php-cli would be configured with all the bells and whistles. I suppose I will start there and compile PHP separately from that with everything.

But, that begs to question. Will xf:import run faster than 1.16 hours? Or would I be wasting an hour trying to build a PHP version that would still crawl along?

 

[Chris D]

If your server has multiple processor cores then you can run the CLI with multiple processes so yes that should be quicker.

An hour isn’t too bad though honestly. Some imports will take 4, 8, 12 or more hours. They’re a pain to do multiple times. An hour not so much.

 

[frm]

If your server has multiple processor cores then you can run the CLI with multiple processes so yes that should be quicker.

Is that a cli setting or an importer setting?

Sounds like I could get it down to 30 minutes... maybe. If I can get a good PHP build that works with it.

 

[Chris D]

It’s in the manual --processes=2 at the end of the command. Replace 2 with the actual number of cores. nproc will tell you if you’re unsure. You can in theory do more than that number but it’s usually a good indicator of the optimal value.

 

[frm]

It’s in the manual --processes=2 at the end of the command. Replace 2 with the actual number of cores. nproc will tell you if you’re unsure. You can in theory do more than that number but it’s usually a good indicator of the optimal value.

Awesome.

Hope I can get a version of php(-cli) that'll run cmd.php it so I can do multiple goes a lot quicker.