vBulletin Big Board Importer [vBulletin 3 + vBulletin 4] [Paid] 1.5.0

[hqarrse]

I thought it worth posting an error and cure that I've come across since the import. I got:

Zend_Feed_Exception: Invalid parameter: author array must include a "name" key with a non-empty string value

which was caused by one of the forum RSS feeds.

The problem was a few hundred threads with an empty username field, and a few thousand posts. The missing information seems to have been a fault of VBulletin and in some cases perhaps of earlier software. The cure I have used, which seems fine so far, is simply to run queries to add a username, eg:

update my_db.xf_thread set username = "error_unknown" where username = "";

Some of the missing usernames do have userids available and a more sophisticated solution would be to check for these, however the posts / threads are old enough and few enough not to worry, and I still have quite a few other fires to put out!

 

[kontrabass]

Just FYI, have successfully imported 52 million records into XenForo 1.3 beta 4 (previously upgraded from 1.2). No errors or issues thus far.

 

[jaymsgq]

I was able to solve the above issues in Export.php , but when I ran the import. php I see these errors

mysql -hlocalhost testdb -uforum_usercp -pjPMHAl -N -q -e " DELETE FROM xf_user_external_auth WHERE user_id > 1 OR provider = 'facebook'; REPAIR TABLE xf_user_external_auth;xf_user_privacyERROR 2 (HY000) at line 1: File '/app/apache/forum/xenbig/importdataxf_user_privacy.txt' not found (Errcode: 2)
(0.01s)...
What is error code 2?

Any advice?

It was a simple error I didn't have a / at the end in dataDir = '/apps/apache/forum/xenbig/importdata/' Hope this helps someone who looking into Error code 2

 

[Naz]

What would cause a 'The requested post could not be found.' error on some posts?

 

[Jake Bunce]

What would cause a 'The requested post could not be found.' error on some posts?

Where do you see that error?

 

[Naz]

Where do you see that error?

If I click in to some threads, I get that. No posts or anything, just the error. Other threads display fine.

 

[Jake Bunce]

If I click in to some threads, I get that. No posts or anything, just the error. Other threads display fine.

When importing against a live vB database it is possible for newly posted threads (made during the import) to have their thread records imported but not their posts.

 

[Naz]

When importing against a live vB database it is possible for newly posted threads (made during the import) to have their thread records imported but not their posts.

Ah, that would make sense. Not sure why that didn't occur to me. Thanks Jake.

 

[Ludachris]

Ughhh, finally got PCNTL extension installed on the server after upgrading PHP. Then upped the memory to 2gb. Now I can at least get the Export.php file to run. But it's getting stuck at the first stage, avatars. I keep getting an "unexpected database error". My ssh program (putty) keeps going inactive even with keepalives turned on.

I made a copy of my live database and am trying to do the test import from the copy. Maybe I'll need to pay someone to do this. Was hoping to be able to do this myself...

 

[Jake Bunce]

Ughhh, finally got PCNTL extension installed on the server after upgrading PHP. Then upped the memory to 2gb. Now I can at least get the Export.php file to run. But it's getting stuck at the first stage, avatars. I keep getting an "unexpected database error". My ssh program (putty) keeps going inactive even with keepalives turned on.

I made a copy of my live database and am trying to do the test import from the copy. Maybe I'll need to pay someone to do this. Was hoping to be able to do this myself...

A detailed db error may be logged in XF:

Admin CP -> Tools -> Server Error Log

You can use 'screen' to deal with SSH timeouts:

http://xenforo.com/community/thread...-3-vbulletin-4-paid.52796/page-15#post-632592

 

[Ludachris]

The db error didn't show up in the error log. It looks like it runs 376.12s on avatars and encounters a db error - "Mysqli prepare error: MySQL server has gone away". Then it goes back to the command line prompt where I can try running the script again. Is the server just running out of memory and crashing?

 

[Jake Bunce]

The db error didn't show up in the error log. It looks like it runs 376.12s on avatars and encounters a db error - "Mysqli prepare error: MySQL server has gone away". Then it goes back to the command line prompt where I can try running the script again. Is the server just running out of memory and crashing?

Mysqli prepare error: MySQL server has gone away

That error is caused by one of these MySQL settings being exceeded:

wait_timeout
max_allowed_packet

Your host or server person needs to increase these settings in MySQL. There is more info here:

http://dev.mysql.com/doc/refman/5.1/en/gone-away.html

 

[Ludachris]

Gotcha, thanks Jake. Finally got both the Export and Import to run... I obviously did something wrong, as both scripts finished within minutes (on a 3.8 million post forum) and absolutely nothing got imported. LOL, this is going to be fun. Time to go back and see what I did wrong.

 

[Ludachris]

Still working my way through this... I think the Export.php worked. But Import.php is not working. It's returning a bunch of instructions about usage. I'm thinking I missed something in the settings for that file. Do either of these lines need to be modified:

if (trim(shell_exec('hostname')) != 'HOSTNAME')

or

$fileDir = realpath(dirname(__FILE__) . '/');

And if so, what's the correct syntax?

I got all the XF db lines updated and the $dataDir path set correctly. What else am I missing?

 

[jaymsgq]

Still working my way through this... I think the Export.php worked. But Import.php is not working. It's returning a bunch of instructions about usage. I'm thinking I missed something in the settings for that file. Do either of these lines need to be modified:

if (trim(shell_exec('hostname')) != 'HOSTNAME')

or

$fileDir = realpath(dirname(__FILE__) . '/');

And if so, what's the correct syntax?

I got all the XF db lines updated and the $dataDir path set correctly. What else am I missing?

Is your both databases on the same server? if its throwing mysql usage edit the Import.php and make sure to use -u and -p see a example below

Example
passthru('mysql -h' . $slaveDbHost . ' ' . $destinationDb . ' -u' . $destDBuser . ' -p' . $destDBpassword . ' -N -q -e "SET NAMES ' . $charset .';LOAD DATA LOCAL INFILE \'' . $file . '\' IGNORE INTO TABLE ' . $table . ' CHARACTER SET ' . $charset .';SELECT @@warning_count;SHOW WARNINGS;"');

 

[Ludachris]

Yes, both are on the same server. I'll see if I can edit that line in the file.

 

[Ludachris]

I found a few lines that look like that, do I need to edit all of them or just one? Can I just add -u and -p in the XF database lines at the top of the file?

 

[Ludachris]

Actually never mind, I added -u and -p in the XF db detail lines and ran it. I'm now getting a ton of "Row doesn't contain data for all columns" errors and the script ends quickly. I found something here in this discussion about extra user profile fields having to be mapped manually. Could that be the cause? We have literally about 50 extra profile fields set up in vb. Will I have to go into XF admincp and replicate those fields first, then map them in the Import.php file?

And I noticed the XF admincp now looks broken. I'm guessing I'll need to restore the db to the saved copy pre-import before I do anything?

 

[jaymsgq]

Actually never mind, I added -u and -p in the XF db detail lines and ran it. I'm now getting a ton of "Row doesn't contain data for all columns" errors and the script ends quickly. I found something here in this discussion about extra user profile fields having to be mapped manually. Could that be the cause? We have literally about 50 extra profile fields set up in vb. Will I have to go into XF admincp and replicate those fields first, then map them in the Import.php file?

And I noticed the XF admincp now looks broken. I'm guessing I'll need to restore the db to the saved copy pre-import before I do anything?

Can you copy the errors here?

 

[Ludachris]

I don't know if I can fit them all here... wonder if I should post them into a txt file and try to upload?