• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

XF 1.1 Importing from IPB3.4 very slow

#1
I'm importing from IPB 3.4 and its still only imported 65% of the members after 12 hours is this normal?

We have 215,711 members to import and over 800,000 posts. Anyway to speed things up ?
 

Jeremy

Well-known member
#2
The software has to iterate over your members, posts, threads, forums, etc. and do a lot of calculations. I haven't imported IPB, but its the nature of a web import.
 

CyclingTribe

Well-known member
#4
I'm importing from IPB 3.4 and its still only imported 65% of the members after 12 hours is this normal?
It sounds slow to me - members should be fairly minimal data and I'd have expected that phase to have completed within 12 hours, unless you're running on some really crappy hardware.

@Mike might be able to advise if he's around, else submit a support ticket.

ISTR that disabling/removing indexes helps - you rebuild them as a last step after all the data has imported - maybe search here on XF.com for prior advice about that.

Cheers,
Shaun :D
 
#5
It sounds slow to me - members should be fairly minimal data and I'd have expected that phase to have completed within 12 hours, unless you're running on some really crappy hardware.

@Mike might be able to advise if he's around, else submit a support ticket.

ISTR that disabling/removing indexes helps - you rebuild them as a last step after all the data has imported - maybe search here on XF.com for prior advice about that.

Cheers,
Shaun :D
Thanks Shaun, Our hardware is pretty good from a good quality host.
 

Walter

Well-known member
#6
Do you know what exactly is slow? Is the CPU under load or do you face heavy I/O?

Have you a Mysql configuration file (usually /etc/my.cnf) that is optimized for Innodb?
 

Jake Bunce

XenForo moderator
Staff member
#8
I'm importing from IPB 3.4 and its still only imported 65% of the members after 12 hours is this normal?
That's way slow.

You might try increasing the batch size:

library/XenForo/Importer/IPBoard.php

Code:
	public function stepUsers($start, array $options)
	{
		$options = array_merge(array(
			'limit' => 100,
			'max' => false,
			// all checkbox options must default to false as they may not be submitted
			'mergeEmail' => false,
			'mergeName' => false,
			'gravatar' => false
		), $options);
A larger batch size can speed things up if your server has enough resources.
 
#9
That's way slow.

You might try increasing the batch size:

library/XenForo/Importer/IPBoard.php

Code:
public function stepUsers($start, array $options)
{
$options = array_merge(array(
'limit' => 100,
'max' => false,
// all checkbox options must default to false as they may not be submitted
'mergeEmail' => false,
'mergeName' => false,
'gravatar' => false
), $options);
A larger batch size can speed things up if your server has enough resources.
I've increased it to 500 but still very slow, Will get some feedback from our host.
 

Jeremy

Well-known member
#10
Sorry, ChrisW, I misread your post. I didn't realize you were talking about members after 12 hours, I thought you meant the whole import. My last post can be disregarded.