Cannot reproduce Threads being imported incorrectly - Duplicate Thread ID's

[Jeff Lange]

[...]
Step 14 of 26: Threads 00:29:40 [3,526] 1.57%
Step 14 of 26: Threads 00:29:50 [3,527] 1.57%
Step 14 of 26: Threads 00:29:59 [3,605] 1.61%
Step 14 of 26: Threads 00:30:08 [3,606] 1.61%


[XF\Db\DuplicateKeyException]
MySQL query error [1062]: Duplicate entry '3992' for key 'PRIMARY'

Not sure what the importer is doing to be honest, it seems to bring in a bunch of much newer threads (giving them low thread id's), and then continues to try to bring in threads that already have those ID's. It's always threads from one specific forum as well. Very odd, like in the thread import at this point it's completed 3606 of the threads, and is now on 3992, but looking in the database, but looking now, somehow there are already threads in there with ID's from 3992 to 4040 that in reality have thread id's in the hundreds of thousands in the source database. They should not have been imported yet, the thread id's inserted don't match the source, and it shouldn't have reached them yet.

I has happened a few times now, seemingly randomly. It previously experienced the issue at thread ID 808, this time it was 3992. It is always threads from the same forum that get imported out of order without their correct thread ID and mess up the threads that actually have that ID.

Jeff

 

[Jeff Lange]

I'm going to look into this further, I suspect it may have something to do with a vB add-on that auto posts threads. Wasn't an issue when doing an XF1 import, but I'll look into that first.

Jeff

 

[Jeff Lange]

For what it's worth, and I don't know exactly why it is happening, I just attempted an import to XF1 and it went smoothly. I'm going to go over to XF1 and then upgrade to XF2, don't want the site down while trying to sort this importer out.

Jeff

 

[Kier]

This sounds like the auto increment ID of your thread table may have been reset at some point, though I couldn't be sure of that.

If you have a thread with a low ID that is newer than a thread with a high ID, you're going to get problems due to the sequential nature of the importer process.

Short of switching everything around so that we import based on posting date rather than ID, this might not be fixable.

 

[Jeff Lange]

The ID's being inserted into the XF2 DB did not match the ID's in the vB database, the ID's were much higher in the vB database but were being inserted incorrectly into the XF2 DB. I should have maybe done some more investigation but at the time I just needed things to work and so I ended up using the XF1 importer which had no issues and then upgraded to XF2.

It's been running well so far.

Jeff