Hard stop during import.

[parafly9]

Hello, I'm doing a test run import from VB4.2 and getting the following error. This was right around step 14 i think. It doesnt look like I am able to resume the import.

I did run an import previously no problem so I'm not sure why it's throwing an error now. Is it possible it's because the forum I am copying from is actively in use?

When I try to restart it immediately throws the same error again.

XF\Db\DuplicateKeyException: MySQL query error [1062]: Duplicate entry '126245' for key 'PRIMARY'
src/XF/Db/AbstractStatement.php:228
Generated by: admin
Nov 3, 2020 at 11:22 AM
Stack trace
INSERT  INTO `xf_thread` (`thread_id`, `node_id`, `title`, `reply_count`, `view_count`, `user_id`, `username`, `post_date`, `sticky`, `discussion_state`, `discussion_open`, `discussion_type`, `type_data`, `first_post_id`, `last_post_date`, `last_post_id`, `last_post_user_id`, `last_post_username`, `first_post_reaction_score`, `first_post_reactions`, `prefix_id`, `custom_fields`, `tags`, `vote_score`, `vote_count`) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
------------

#0 src/XF/Db/Mysqli/Statement.php(196): XF\Db\AbstractStatement->getException('MySQL query err...', 1062, '23000')
#1 src/XF/Db/Mysqli/Statement.php(77): XF\Db\Mysqli\Statement->getException('MySQL query err...', 1062, '23000')
#2 src/XF/Db/AbstractAdapter.php(94): XF\Db\Mysqli\Statement->execute()
#3 src/XF/Db/AbstractAdapter.php(218): XF\Db\AbstractAdapter->query('INSERT INTO `x...', Array)
#4 src/XF/Import/Data/EntityEmulator.php(327): XF\Db\AbstractAdapter->insert('xf_thread', Array)
#5 src/XF/Import/Data/AbstractEmulatedData.php(71): XF\Import\Data\EntityEmulator->insert(126245, Object(XF\Db\Mysqli\Adapter))
#6 src/XF/Import/Data/AbstractData.php(127): XF\Import\Data\AbstractEmulatedData->write(126245)
#7 src/addons/XFI/Import/Importer/vBulletin.php(3041): XF\Import\Data\AbstractData->save(126245)
#8 src/XF/Import/Runner.php(242): XFI\Import\Importer\vBulletin->stepThreads(Object(XF\Import\StepState), Array, 8)
#9 src/XF/Import/Runner.php(59): XF\Import\Runner->runStep('threads', Object(XF\Import\StepState), 8)
#10 src/XF/Admin/Controller/Import.php(234): XF\Import\Runner->run()
#11 src/XF/Mvc/Dispatcher.php(350): XF\Admin\Controller\Import->actionRun(Object(XF\Mvc\ParameterBag))
#12 src/XF/Mvc/Dispatcher.php(257): XF\Mvc\Dispatcher->dispatchClass('XF:Import', 'Run', Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Import), NULL)
#13 src/XF/Mvc/Dispatcher.php(113): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Import), NULL)
#14 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#15 src/XF/App.php(2300): XF\Mvc\Dispatcher->run()
#16 src/XF.php(465): XF\App->run()
#17 admin.php(13): XF::runApp('XF\\Admin\\App')
#18 {main}
Request state
array(4) {
["url"] => string(21) "/admin.php?import/run"
["referrer"] => string(50) "https://testforum.alpinezone.com/admin.php?import/"
["_GET"] => array(1) {
["import/run"] => string(0) ""
}
["_POST"] => array(1) {
["_xfToken"] => string(8) "********"
}
}

 

[parafly9]

Tried re-running in CLI and get the same error.

Coming from regular vb 4.2.5 install. Also I realize I do have "retain content ID's" selected, which I didn't have previously when i was able to do a successful import. I will try again without retain content ID's.

 

[Paul B]

You can't import into a forum with existing content and retain IDs, as the IDs will clash.

 

[parafly9]

You can't import into a forum with existing content and retain IDs, as the IDs will clash.

There is no existing content, it's a blank install. only the admin created during setup. The previous test import was a completely different xenforo install on a different domain.

 

[Paul B]

I did run an import previously no problem so I'm not sure why it's throwing an error now.

I assumed you already had content due to the above.

In that case, it suggests there may be content in the source database with duplicate IDs.

Check how many threads there are with an ID of 126245.

 

[parafly9]

I assumed you already had content due to the above.

In that case, it suggests there may be content in the source database with duplicate IDs.

Check how many threads there are with an ID of 126245.

Hmm. I checked and I'm only getting a single result.

 

[Paul B]

It could have been a race condition.

Can you restore the clean backup taken before doing any imports and try again with the retain IDs option checked.

 

[parafly9]

It could have been a race condition.

Can you restore the clean backup taken before doing any imports and try again with the retain IDs option checked.

Trying again, this time with a copy of the database that isn't in use. maybe that was the problem.

 

[Paul B]

It's always recommended to close the site or use a copy of the DB as importing from an active site could result in orphaned data.

 

[parafly9]

It's always recommended to close the site or use a copy of the DB as importing from an active site could result in orphaned data.

That makes sense. I'm running at 55% of threads done right now so so far so good.

 

[parafly9]

Well so far so good (knock on wood), I'm all the way through threads and onto posts now and so far no issues.

 

[parafly9]

Spoke too soon. 99.75% done and blam.





Because it's a DB copy for ****s & giggles I tried to delete this post 1052109, and it threw the same error for post 1052108, deleted that, same error again for 1052110. can't seem to get past it.

 

[parafly9]

I stopped and tried again, tried in CLI, made sure board was closed (it's a test site anyway), importing from a database copy (not active).

I keep getting the same errors, but they seem to be occuring somewhere different (i.e. a different line or entry).

Anything else I can do to try to get this to push through?

 

[Paul B]

As it happens randomly with different steps and IDs each import, I can only assume it's some sort of race condition or an issue at the server level (e.g. resources).

Is the server particularly busy, overloaded?

 

[parafly9]

As it happens randomly with different steps and IDs each import, I can only assume it's some sort of race condition or an issue at the server level (e.g. resources).

Is the server particularly busy, overloaded?

I don't believe the server is overloaded, but i am asking the host. I have a vps with 8gb memory 4vcpu, and my site isn't particularly busy at the moment.

 

[parafly9]

Host came back with this:

Thanks for reaching out. I have adjusted a few limits in terms of the memory of PHP and compiled mod_lsapi into your server which allows for faster PHP processing times and better caching. The server itself looks to be operational and responding without issue but I do see the database connection dropping. Can you try it again and let us know if the changes we have made have helped things at all.

In case it helps the team that you are working with to help find a solution here is the configuration file for your MySQL.

[mysqld]
sql_mode=NO_ENGINE_SUBSTITUTION
performance-schema=0
innodb_file_per_table
max_connections=500
max_user_connections=30
wait_timeout=60
connect_timeout=60
max_allowed_packet=268435456
max_connect_errors=100
flush
flush_time=60

performance-schema=0
default-storage-engine=MyISAM
open_files_limit=40000
[mysql.server]
user=mysql

[mysqld_safe]

[mysqldump]
quick
max_allowed_packet=16M

[myisamchk]
key-buffer-size=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M

[mysqlhotcopy]
interactive-timeout

 

[parafly9]

well, I got a completion this morning finally. Not sure what was different but it went all the way to the end. I did delete the "main forum" default forum that is loaded in, I'm not sure if that has anything to do with it.

 

[Paul B]

Deleting that forum won't have had any effect.

It sounds like the changes your host made is what resolved it.