XF 2.2 Attachment Import

[Seeker-Smith]

Any idea how long it'll take to import 300K attachments from vb5? Also will it be possible to merge more content later?

 

[Paul B]

Impossible to say - it depends on server spec and set up.

If by "merge more content later" you mean do a top up import, no, that's not possible.

Have a read of this guide: https://xenforo.com/community/threads/importing-guidelines.25325/

 

[Seeker-Smith]

Thanks. Yes I've read the guide but need to do it again. For today I'm setting up a test site where I can get more familiar with the db, and other things. I ran the demo a couple times and the permissions at first were different than I'm use to but I see the ease and power.

 

[niktator]

In the test import from vb5 i used 24 threads and everything was beeing imported very fast. like 3 hours. BUT the attachments take ages! nah centurys. I guess the --processes=x is not working for attchments? I deleted 75% of my forums including attachments to thest the import (i deleted the picture heavy ones) - and only the process for the images took more then 11 hours.
Its there any spacial command for having more attchments beeing imported? I got 6,8gb/s write speed and 64 cores + 256gb ram on that machine - and the hardware was on idel when doing attchments ;/

 

[Seeker-Smith]

You need to use the CLI Importer. The browser would have taken maybe a week in my case and even then I'd have to restart often.

I did 300,00 attachments in 4.5 hours.

 

[niktator]

I did use the CLI importer. 2,268,417 attachments took 11 hours. But if I want to convert my life system, it will be more then 4,3 millon.
The point was: why isnt this handled parallel like the rest of the migration?

 

[Seeker-Smith]

I can see 11 hours for that many attachments for sure. I'm not sure what a life system is or the rest of your comment. I tried to go from vb4.25 to mybb and it took 5 days only to end with most attachments not importing.

 

[niktator]

Life System is the system i finally plan to migrate. As I wrote I deleted alot of posts including pictures to check if and how the import is working at all.
Now I got the problem, that my downtime would be way over a day if i can`t speed up the attachment import.

 

[Seeker-Smith]

I feel your pain but migration does take time. In my case I tested it then closed my business and did it. I'm not sure if there is any other option. With 2 million attachments it's going to take time. The rest should be pretty quick.

 

[Seeker-Smith]

Not only that but you'll need to monitor the progress and restart if needed. It's not a job I want to do again.

 

[niktator]

For anyone who is interested: The solution is NOT to go for 1 core - and it`s not going for all cores.
It IS kinda parallel but if you got too many processes, the db will wait and lock.
I tested on some 24 core/threads server and the sweet spot is:

php7.4 cmd.php xf:import --processes=7

So for 24 cores the sweet spot is 7 for attachments.