XF 2.1 In Browser - Horrible Import Speed

blablafixer

Active member
The import has been running for 8 hours so far and for whatever reason has slowed down tenfold towards the end. Not only that but if the browser window goes unresponsive the import stops.

It uses almost none of the available system resources, I've seen it use 25% of cpu usage max and 50MB/s disk read at max, it should be using much more resources to get it all over with. I don't want to cancel and do command line at this point but I'm sure it would be quicker.

The in-browser should be just a viewer when it's doing the user import step and not command the process every 8 seconds as it does now. There is no UI for cancelling the import so there's no reason for the browser to not just be a viewer during this step.

Should I edit some file to increase the "POST" speed from 8 seconds to 1 second to make it run faster?
 

Chris D

XenForo developer
Staff member
There is some overhead with running imports (and any long running tasks) in the browser.

Unfortunately the very mechanics of how the browser import works cannot be changed. It has to work on a principle of refreshing the page every X otherwise your web server would likely kill the request after some period of time and there would be no indication of progress.

There's really no concept of the browser just being "a viewer". The server side processing requires physical activity on the server for its scripts to run, the only way of doing that is by refreshing the page periodically as we do currently, as you noticed, every 8 seconds.

You can change the default using src/config.php:

PHP:
$config['jobMaxRunTime'] = 30;

This may reduce some of the overhead, but it will be at the expense of fewer progress updates.

It is possible to switch to the command line without losing any progress. You would just close the browser window so progress stops momentarily, and then run the import command from the command line:

Code:
php cmd.php xf:import

If you have spare system resources, as it appears you do, you can allow the command line to run multiple processes:

Code:
php cmd.php xf-import --processes=4

This will resume the import from where the browser left off.

This still uses the jobMaxRunTime value so you may still wish to increase that.
 

blablafixer

Active member
Hey @Chris D thanks for your help! I did notice the progress stopped when my computer when to sleep.

Also, I tried to close the tab and do the command line however:

php cmd.php xf:import
This import has been started through another method.

So that didn't work. I have a further question, if I revisit the import page I'm met with this:

import.png

Does "Restart" mean continue where it left off? Or start over from scratch? The way I continue right now is by hitting back in my browser, because otherwise I am met with that page above.

I added the config entry you suggested but it didn't improve the import speed.
 

Chris D

XenForo developer
Staff member
Oops. I'm sorry, I forgot there was a restriction on resuming the import via another method.

Yeah it will continue where it left off if you click "Restart import".
 
Top