XF 1.1 Importing from IPB3.4 very slow

ChrisW

Member
I'm importing from IPB 3.4 and its still only imported 65% of the members after 12 hours is this normal?

We have 215,711 members to import and over 800,000 posts. Anyway to speed things up ?
 
The software has to iterate over your members, posts, threads, forums, etc. and do a lot of calculations. I haven't imported IPB, but its the nature of a web import.
 
I'm importing from IPB 3.4 and its still only imported 65% of the members after 12 hours is this normal?

It sounds slow to me - members should be fairly minimal data and I'd have expected that phase to have completed within 12 hours, unless you're running on some really crappy hardware.

@Mike might be able to advise if he's around, else submit a support ticket.

ISTR that disabling/removing indexes helps - you rebuild them as a last step after all the data has imported - maybe search here on XF.com for prior advice about that.

Cheers,
Shaun :D
 
It sounds slow to me - members should be fairly minimal data and I'd have expected that phase to have completed within 12 hours, unless you're running on some really crappy hardware.

@Mike might be able to advise if he's around, else submit a support ticket.

ISTR that disabling/removing indexes helps - you rebuild them as a last step after all the data has imported - maybe search here on XF.com for prior advice about that.

Cheers,
Shaun :D

Thanks Shaun, Our hardware is pretty good from a good quality host.
 
Do you know what exactly is slow? Is the CPU under load or do you face heavy I/O?

Have you a Mysql configuration file (usually /etc/my.cnf) that is optimized for Innodb?
 
I'm importing from IPB 3.4 and its still only imported 65% of the members after 12 hours is this normal?

That's way slow.

You might try increasing the batch size:

library/XenForo/Importer/IPBoard.php

Rich (BB code):
	public function stepUsers($start, array $options)
	{
		$options = array_merge(array(
			'limit' => 100,
			'max' => false,
			// all checkbox options must default to false as they may not be submitted
			'mergeEmail' => false,
			'mergeName' => false,
			'gravatar' => false
		), $options);

A larger batch size can speed things up if your server has enough resources.
 
That's way slow.

You might try increasing the batch size:

library/XenForo/Importer/IPBoard.php

Rich (BB code):
public function stepUsers($start, array $options)
{
$options = array_merge(array(
'limit' => 100,
'max' => false,
// all checkbox options must default to false as they may not be submitted
'mergeEmail' => false,
'mergeName' => false,
'gravatar' => false
), $options);

A larger batch size can speed things up if your server has enough resources.

I've increased it to 500 but still very slow, Will get some feedback from our host.
 
Sorry, ChrisW, I misread your post. I didn't realize you were talking about members after 12 hours, I thought you meant the whole import. My last post can be disregarded.
 
Top Bottom