XF 1.1 IPB Import - Server error

guvner · Feb 2, 2012

Hi guys - first post, pity it's an error that I can't figure out.

I have an IPB forum that was VB 4x until January 2011.

I was hoping to let my staff and myself give XF a thorough workout by importing my IPB database (325,000 posts & 85,000 members) into a test installation in a sub folder on my forum's domain.

Problem is the import is stalling quite early on at the "Import Custom User Fields" point. The error I'm seeing is:

Server Error

Undefined offset: 1

XenForo_Application::handlePhpError() in XenForo/Importer/IPBoard.php at line 665
XenForo_Importer_IPBoard->_convertUserFieldChoices() in XenForo/Importer/IPBoard.php at line 589
XenForo_Importer_IPBoard->stepUserFields() in XenForo/Importer/Abstract.php at line 77
XenForo_Importer_Abstract->runStep() in XenForo/ControllerAdmin/Import.php at line 180
XenForo_ControllerAdmin_Import->_runStep() in XenForo/ControllerAdmin/Import.php at line 232
XenForo_ControllerAdmin_Import->_startStep() in XenForo/ControllerAdmin/Import.php at line 175
XenForo_ControllerAdmin_Import->actionStartStep() in XenForo/FrontController.php at line 310
XenForo_FrontController->dispatch() in XenForo/FrontController.php at line 132
XenForo_FrontController->run() in /home/xxx/public_html/xfdev/admin.php at line 13

Having researched similar errors here I've double checked folder permissions and both data and internal_data are set to 777.

Jake mentioned in another thread it might be an account ownership issue, since I'm on a dedicated server and the original conversion required some custom work and as a result was done by IPB staff, I guess it's possible that ownership of the account was changed and I'm not aware of it, but if that's so I'm unsure first how to verify it, and secondly how to correct it.

Any help would be much appreciated.
Thanks,
Mike

Jake Bunce · Feb 2, 2012

guvner said:
Undefined offset: 1

XenForo_Application::handlePhpError() in XenForo/Importer/IPBoard.php at line 665

That line is this code:

Rich (BB code):

	protected function _convertUserFieldChoices(array $profileField)
	{
		$choices = array();

		foreach (explode('|', $this->_convertToUtf8($profileField['pf_content'])) AS $choiceString)
		{
			list($key, $value) = explode('=', $choiceString, 2);

			$choices[$key] = $value;
		}

		return $choices;
	}

This looks like a problem with the source data. Or at least the data is not what the importer expects. The nature of the error is that the importer is trying to extract two values from $choiceString but it is apparently only getting one or none.

How big is your database? I can take a look if you give me a copy of your database. Or access to phpmyadmin would also work.

guvner · Feb 2, 2012

Hi Jake,

When I was with VB I had a range of custom profile fields setup to enable users to detail the hardware components/OS used in their media center PCs. Those fields never worked after the conversion to IPB although I believe they are still there. Maybe that's it?

The DB size is currently 1277.78 MB and PHPMyAdmin reports no errors.

Rather than mess with a large SQL file I just sent you login details - thank you!

Jake Bunce · Feb 3, 2012

I examined your IPB database. pfields_data.pf_content is empty for some records where it shouldn't be empty. That is the problem. I was able to fix this by adding the red code:

library/XenForo/Importer/IPBoard.php

Rich (BB code):

					if ($profileField['pf_input_format'])
					{
						$import['match_type'] = 'regex';
						$import['match_regex'] = $this->_convertUserFieldMatchTypeToRegex($profileField['pf_input_format']);
					}

					if ($convertChoices AND $profileField['pf_content'])
					{
						$import['field_choices'] = $this->_convertUserFieldChoices($profileField);
					}

					if ($newFieldId = $model->importUserField($profileField['pf_id'], $import))
					{
						$total++;
					}
					break;

The custom fields module runs fine now. I wanted to continue the import for further testing but I encountered a database error on the very next module (the user module):

Mysqli statement execute error : Incorrect key file for table '/tmp/#sql_1049_0.MYI'; try to repair it

I ran a repair on both databases. No errors were returned, but the database error persists in the user module. It's weird. I suggest restarting the server. If you don't have that level of access then contact the host.

guvner · Feb 3, 2012

Thanks Jake - I appreciate your help.

I run a dedicated server so I'm able to use WHM to perform a graceful reboot - I also have root access via the shell.

Yesterday before I tried the import for the first time I turned off my forum and whilst backing up my database via phpMyAdmin my server became slow and partially unresponsive even though it's a pretty powerful box.

I terminated the backup and since the forum was already closed I took the opportunity to do a graceful reboot to flush out any gremlins. I then ran the backup from the shell and all was OK.

Would you suggest another graceful reboot or would using the remote reboot port be required?

Jake Bunce · Feb 3, 2012

Yeah do another graceful.

guvner · Feb 3, 2012

OK well sadly that didn't work.

I rebooted, reinstalled (same user/pw combo that you used) and tried again.

It stalled at the user import with:

Code:

Mysqli statement execute error : Incorrect key file for table '/tmp/#sql_104b_0.MYI'; try to repair it
 
Zend_Db_Statement_Mysqli->_execute() in Zend/Db/Statement.php at line 297
Zend_Db_Statement->execute() in Zend/Db/Adapter/Abstract.php at line 479
Zend_Db_Adapter_Abstract->query() in Zend/Db/Adapter/Abstract.php at line 734
Zend_Db_Adapter_Abstract->fetchAll() in XenForo/Importer/IPBoard.php at line 710
XenForo_Importer_IPBoard->stepUsers() in XenForo/Importer/Abstract.php at line 77
XenForo_Importer_Abstract->runStep() in XenForo/ControllerAdmin/Import.php at line 180
XenForo_ControllerAdmin_Import->_runStep() in XenForo/ControllerAdmin/Import.php at line 232
XenForo_ControllerAdmin_Import->_startStep() in XenForo/ControllerAdmin/Import.php at line 175
XenForo_ControllerAdmin_Import->actionStartStep() in XenForo/FrontController.php at line 310
XenForo_FrontController->dispatch() in XenForo/FrontController.php at line 132
XenForo_FrontController->run() in /home/xxxxx/public_html/xfdev/admin.php at line 13

When I Google "Mysqli statement execute error : Incorrect key file for table '/tmp/#sql" I get a few hits and one of the more interesting ones is here - http://www.mysqlperformancetuning.com/a-fix-for-incorrect-key-file-for-table-mysql

-specifically:"The idea of it being suggested that one needs to run a repair on a temporary table should raise an eyebrow. It just doesn't make much sense: it suggests it's somehow corrupted, but since it's transitory and has just been freshly created by MySQL, it should never be needed."

I did a phpMyAdmin search on "104b" and got hits in the Twitter connect, user and post tables. Given that it's the user import that stalls I went through that table and looked for any user entries that seemed wrong.

Found a few that looked to have strange entries and after checking their register date, last active date and post count and verifying they've never been active I deleted the\m all from the IPB admincp.

The tables were flushed but the error continues. Maybe I'm searching the DB for the wrong key?

I'm a little lost at this point - any ideas?

Paul B · Feb 3, 2012

Just an idea, but do you have enough room on the disk/in the partition for these temp tables?

Although I would expect the error to be different if that was the case.

guvner · Feb 3, 2012

Hi Brogan

I'd already had a look at that and disk space doesn't appear to be an issue:

/dev/sda1 99M 24M 71M 25% /boot
/dev/sda2 451G 52G 376G 13% /
/dev/sdb1 452G 71G 358G 17% /disk1
/usr/tmpDSK 485M 11M 449M 3% /tmp

The table didn't render the headings but the % numbers above refer to the space used which as you can see are small numbers.

If mysql is trying to allocate temp disk space to run the process and then reporting that the tables it creates are corrupt and need repair - is there any possibility it's a problem with the IPB import script?

Jake Bunce · Feb 4, 2012

I have actually never encountered this kind of problem before where the error indicates corruption but a repair comes back clean. It is definitely a problem with MySQL and not the import script.

You could try running a repair in the mysql prompt using the shell, but I don't expect that to give a different result than phpmyadmin.

guvner · Feb 4, 2012

I'll give it a go - I'll try first to run a repair on the whole database and if that dosen't fix it I'll try and target that specific table.

Are you able to give me the exact table name to use? I've already tried finding the table using various combinations of the string in the error but I'm not sure which is the exact table name.

Incorrect key file for table '/tmp/#sql_104b_0.MYI';

Jake Bunce · Feb 4, 2012

The IPB tables that are included in that query are:

members
pfields_content
profile_portal
admin_permission_rows

(with your prefix)

guvner · Feb 9, 2012

Geez - nearly a week later... I wish my real-life job didn't keep getting in the way of the important stuff!

OK after closing the forum and backing up my DB I first ran a standard repair through the shell - no change.

Then I got a little braver, shut down mysql and ran a myisamchk repair - and everything turned to crap. I brought down my main forum with what appeared to be a large number of DB errors.

Anyhow, a quick DB restore and all was good again - scary though.

To avoid messing with my live DB, I duplicated my main site on a spare domain and installed XF next to it in another directory.

For the import I used your modified IPBoard.php and all went OK until (once again) I got to the Members and got a similar error:

Mysqli statement execute error : Incorrect key file for table '/tmp/#sql_5394_0.MYI'; try to repair it

So now I'm really stuck - any ideas what I can do to get around this problem or am I stuck with IPB forever?

Jake Bunce · Feb 10, 2012

What version of MySQL (use the "mysql -V" command)? Possible bug:

http://bugs.mysql.com/bug.php?id=12291

There is also a report of this error happening when /tmp gets full, so you should check your partitions for available space (use the "df" command):

http://bugs.mysql.com/bug.php?id=24413

guvner · Feb 10, 2012

Thanks Jake - glad you're still there.

OK ran those commands, results as follows:

Can't see a problem with the tmp space unless I'm reading something wrong.

I didn't realise I was so far behind with my MySQL version - seems that the latest stable version is 5.5.20.

Do you think that might be connected?

Jake Bunce · Feb 10, 2012

Yeah you may want to upgrade MySQL. Be sure to backup all of your databases before the upgrade though.

Regarding the /tmp directory... it's a question of whether or not that partition gets full during the query. If you have extra storage space then it doesn't hurt to increase the size.

guvner · Feb 10, 2012

I'll upgrade MySQL once I've backed up all databases on my server.

Not sure which command to use to increase space for /usr/tmpDSK though - can you help please? If it's a full on partition job I'll probable have to get the server techs to do it so I don't break anything.

Jake Bunce · Feb 10, 2012

Probably let your server guy do it.

http://antg.org/howto/2010/04/10/37/increase-the-size-of-tmp-usrtmpdsk-partition-in-linux.html

guvner · Feb 10, 2012

LOL - after reading that - good call !

Final question on this part - given the requirements of the XF import script which of course I don't understand, what would be the best final size of the temp partition to take that issue off the table?

Thanks.

Jake Bunce · Feb 10, 2012

Try 2GB.

XF 1.1 IPB Import - Server error

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

XenForo moderator

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Member

Well-known member

Similar threads

We value your privacy