Resource icon

vBulletin Big Board Importer [vBulletin 3 + vBulletin 4] [Paid] 1.5.0

No permission to buy ($150.00)
Overview Updates (9) Reviews (3) History Discussion
Yep, definitely something wrong. I don't know what that error is from but when I moved my site with 80k users, I had just over 1 hr of downtime from the time I started the export (I had already exported the attachments so it only did an incremental export of those, cutting the export time a lot) to the time I had the forums/users/permissions rebuilt and the site back online. At that point I was doing rebuilds of other caches but the site was basically functional.

Plus, my system specs were dramatically worse than yours.
 
Could you please update this to work with XF 1.4? I paid $150 for this add-on (the most expensive on the Resources section of XenForo.com) and I was expecting a better support in terms of updates... (n)
 
It requires manual edits. They should update this add-on and release a new version including the required changes...
 
Super120 said:
It requires manual edits. They should update this add-on and release a new version including the required changes...
Click to expand...

Excuse me? There is no obligation for us to do anything. Telling resource authors what they should do will not win you any friends.

The solution in the meanwhile has been posted by Jake. An update will be released when we see fit.
 
Ok, I'm going again with a clean install - trying to make it error free this time. Just running the export and I've 356 pages of this error:

Code: 
ErrorException: Undefined offset: 8 - Export.php:361
Generated By: Unknown Account, 20 minutes ago
Stack Trace
#0 /var/www/vhosts/***/test/Export.php(361): XenForo_Application::handlePhpError(8, 'Undefined offse...', '/var/www/vhosts...', 361, Array)
#1 /var/www/vhosts/***/test/Export.php(1906): DigitalPointExporter->users()
#2 {main}
Request State
array(3) {
  ["url"] => string(7) "http://"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}

So it's generating an error exporting users - I'm on vBulletin 4.1.9 (rather than latest version). I might have quite a few users who have commas and apostrophes in their names (although with only 25k users I'm not sure I'd have enough of those to throw up 356 pages of errors - when I did the standard XF import I only had to manually fix about 30 users). I can't think of anything else that could be different about my installation.
 
Superfluous said:
Ok, I'm going again with a clean install - trying to make it error free this time. Just running the export and I've 356 pages of this error:

Code: 
ErrorException: Undefined offset: 8 - Export.php:361
Generated By: Unknown Account, 20 minutes ago
Stack Trace
#0 /var/www/vhosts/***/test/Export.php(361): XenForo_Application::handlePhpError(8, 'Undefined offse...', '/var/www/vhosts...', 361, Array)
#1 /var/www/vhosts/***/test/Export.php(1906): DigitalPointExporter->users()
#2 {main}
Request State
array(3) {
  ["url"] => string(7) "http://"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}

So it's generating an error exporting users - I'm on vBulletin 4.1.9 (rather than latest version). I might have quite a few users who have commas and apostrophes in their names (although with only 25k users I'm not sure I'd have enough of those to throw up 356 pages of errors - when I did the standard XF import I only had to manually fix about 30 users). I can't think of anything else that could be different about my installation.
Click to expand...

That means you didn't map all of your group IDs. You need to revisit this setting in the Export file:

Code: 
// MAP VB GROUP IDS TO XF GROUP IDS
	public static $groupMap = array(
				1 => 1,
				2 => 2,
				3 => 1,
				4 => 1,
				5 => 4,
				6 => 3,
				7 => 4,
				11 => 5,
				13 => 6, 
				28 => 7,
				29 => 8,
				30 => 9,
				31 => 10, 
				32 => 11,
				33 => 12,
				35 => 13,
				40 => 14,
				42 => 15,
				43 => 16,
				44 => 17,
				45 => 18,
			);
 
Jake Bunce said:
That means you didn't map all of your group IDs. You need to revisit this setting in the Export file:

Code: 
// MAP VB GROUP IDS TO XF GROUP IDS
    public static $groupMap = array(
                1 => 1,
                2 => 2,
                3 => 1,
                4 => 1,
                5 => 4,
                6 => 3,
                7 => 4,
                11 => 5,
                13 => 6,
                28 => 7,
                29 => 8,
                30 => 9,
                31 => 10,
                32 => 11,
                33 => 12,
                35 => 13,
                40 => 14,
                42 => 15,
                43 => 16,
                44 => 17,
                45 => 18,
            );
Click to expand...

Thanks, I suspected that and checked my mappings which seem to be correct (on vb backend vs 4 options on XF backend) - although I didn't delete the mappings that weren't needed (such as 45=>18) and I didn't include mappings that no users were a member.

Also note the code, handlePhpError(8, 'Undefined offse...' - does that mean that it's usergroup 8 which is the issue, which is banned users and which was defined in the separate variable?
 
I defined banned users 8 and put them into registered users, and the script runs in about 5 seconds, and has no errors (previously it took about 10 minutes), so I found my problem.

I'm thinking now that my imports and rebuilds might be slow because they have been throwing up errors.
 
Ok, after 2nd import finished I'm getting this error:

Code: 
Error Info
ErrorException: unserialize(): Error at offset 0 of 14 bytes - library/XenForo/Model/Forum.php:253
Generated By: Unknown Account, 16 minutes ago
Stack Trace
#0 [internal function]: XenForo_Application::handlePhpError(8, 'unserialize(): ...', '/var/www/vhosts...', 253, Array)
#1 /var/www/vhosts/***/test/library/XenForo/Model/Forum.php(253): unserialize('last_post_date')
#2 /var/www/vhosts/***/test/library/XenForo/ControllerHelper/ForumThreadPost.php(55): XenForo_Model_Forum->prepareForum(Array)
#3 /var/www/vhosts/***/test/library/XenForo/ControllerHelper/ForumThreadPost.php(77): XenForo_ControllerHelper_ForumThreadPost->assertForumValidAndViewable(2, Array)
#4 /var/www/vhosts/***/test/library/XenForo/ControllerPublic/Thread.php(32): XenForo_ControllerHelper_ForumThreadPost->assertThreadValidAndViewable(192, Array, Array)
#5 /var/www/vhosts/***/test/library/XenForo/FrontController.php(347): XenForo_ControllerPublic_Thread->actionIndex()
#6 /var/www/vhosts/***/test/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#7 /var/www/vhosts/***/test/index.php(13): XenForo_FrontController->run()
#8 {main}
Request State
array(3) {
  ["url"] => string(82) "http://test.***/index.php?threads/5s-the-night-1st-time-in-over-a-year.192/"
  ["_GET"] => array(1) {
    ["threads/5s-the-night-1st-time-in-over-a-year_192/"] => string(0) ""
  }
  ["_POST"] => array(0) {
  }
}

I'm assuming it's some sort of datetime format issue (I'm using British format on the VB forum) - anyone any ideas?
 
Superfluous said:
Ok, after 2nd import finished I'm getting this error:

Code: 
Error Info
ErrorException: unserialize(): Error at offset 0 of 14 bytes - library/XenForo/Model/Forum.php:253
Generated By: Unknown Account, 16 minutes ago
Stack Trace
#0 [internal function]: XenForo_Application::handlePhpError(8, 'unserialize(): ...', '/var/www/vhosts...', 253, Array)
#1 /var/www/vhosts/***/test/library/XenForo/Model/Forum.php(253): unserialize('last_post_date')
#2 /var/www/vhosts/***/test/library/XenForo/ControllerHelper/ForumThreadPost.php(55): XenForo_Model_Forum->prepareForum(Array)
#3 /var/www/vhosts/***/test/library/XenForo/ControllerHelper/ForumThreadPost.php(77): XenForo_ControllerHelper_ForumThreadPost->assertForumValidAndViewable(2, Array)
#4 /var/www/vhosts/***/test/library/XenForo/ControllerPublic/Thread.php(32): XenForo_ControllerHelper_ForumThreadPost->assertThreadValidAndViewable(192, Array, Array)
#5 /var/www/vhosts/***/test/library/XenForo/FrontController.php(347): XenForo_ControllerPublic_Thread->actionIndex()
#6 /var/www/vhosts/***/test/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#7 /var/www/vhosts/***/test/index.php(13): XenForo_FrontController->run()
#8 {main}
Request State
array(3) {
  ["url"] => string(82) "http://test.***/index.php?threads/5s-the-night-1st-time-in-over-a-year.192/"
  ["_GET"] => array(1) {
    ["threads/5s-the-night-1st-time-in-over-a-year_192/"] => string(0) ""
  }
  ["_POST"] => array(0) {
  }
}

I'm assuming it's some sort of datetime format issue (I'm using British format on the VB forum) - anyone any ideas?
Click to expand...

Did you rebuild your forum info after the import?

Admin CP -> Tools -> Rebuild Caches
 
I have a problem with the newest version of the importer (1.4) and attachments:

1470 An unexpected database error occurred. Please try again later.
<!-- Mysqli prepare error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' , 1)' at line 4 -->root@********


Zend_Db_Statement_Mysqli_Exception: Mysqli prepare error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' , 1)' at line 4 - library/Zend/Db/Statement/Mysqli.php:77
Generiert durch: Unbekanntes Benutzerkonto, Vor 6 Minuten
Stapelverfolgung

#0 /home/*****/public_html/eos/library/Zend/Db/Statement.php(115): Zend_Db_Statement_Mysqli->_prepare('???????INSERT I...')
#1 /home/*****/public_html/eos/library/Zend/Db/Adapter/Mysqli.php(381): Zend_Db_Statement->__construct(Object(Zend_Db_Adapter_Mysqli), '???????INSERT I...')
#2 /home/*****/public_html/eos/library/Zend/Db/Adapter/Abstract.php(478): Zend_Db_Adapter_Mysqli->prepare('???????INSERT I...')
#3 /home/****/public_html/eos/Export.php(1388): Zend_Db_Adapter_Abstract->query('???????INSERT I...')
#4 /home/*****/public_html/eos/Export.php(1901): DigitalPointExporter->attachments()
#5 {main}

The reason for the error is this statement:
Code: 
                                    INSERT INTO xf_attachment_data
                                                (user_id, upload_date, filename, file_size, file_hash, width, height, thumbnail_width, thumbnail_height, attach_count)
                                        VALUES
                                                (11956, 1128882943, "HC_HeavenlyBridge.gif", 41890, "b565edbad937b52d1a7882a69dc2168f", 1, 1, , , 1)

There are some attachments in the database where the importer fails to set thumbnail_width and _height.
 
Last edited:
Jake Bunce said:
Did you rebuild your forum info after the import?

Admin CP -> Tools -> Rebuild Caches
Click to expand...

Short answer yes...

Long answer - there are lots of problems with the installation (and I apologise for hitting you with it all at once...).

My 'cleaning up' process seems to take longer than the import process - I started rebuilding caches during the cleaning up process because it took so long.

As you can see almost twice as many seconds for the cleaning up process as the Import Total Time (nearly 7 hours for clean up). I checked the import log, there are about 7.2 million rows, so it kinda makes sense that it takes almost as long as importing the almost 7 million rows of posts - but I'd be surprised if it was supposed to take this long (the fact I haven't seen anyone else complain about it).

1vvAR.jpg


I get quite a few of these type errors (incorrect integer values, and doesn't contain data for all columns:

M0Yu7.jpg


Looking through the database tables I'm thinking that there were missing columns in the insert queries (assuming column names weren't defined for inserts) - for instance look at the users table where it appears avatar height/width values have found their way into the avatar_date column

26trx.jpg


So even after trying to rebuild caches and playing about for ages, my administrator account doesn't have permission to view forums, there are no avatars, the post counts are all wrong in the members section, and the number of members is wrong in the side bar.

Re5nh.jpg


FTDjN.jpg


The Xenforo default importer worked perfectly (completely pain free) - it just took 24 hours alone for the posts to import. So I'm thinking it's either my configuration for the big importer - or the default importer defines column names during inserts, but the big importer doesn't - so my export database (on vb 4.1.9) has a different table structure to what is expected.

Also, something which I think might be a side issue - I'm getting a permissions error when trying to rebuild attachment thumbnails (it's not obvious from the error, is it a chmod issue?)

Code: 
Error Info
XenForo_Exception: Thumb rebuild for #20611: Failed to write the attachment thumbnail data. - library/XenForo/DataWriter/AttachmentData.php:168
Generated By: MalteseMick, Yesterday at 11:33 PM
Stack Trace
#0 /var/www/vhosts/test/library/XenForo/DataWriter.php(1409): XenForo_DataWriter_AttachmentData->_postSave()
#1 /var/www/vhosts/test/library/XenForo/Deferred/AttachmentThumb.php(59): XenForo_DataWriter->save()
#2 /var/www/vhosts/test/library/XenForo/Model/Deferred.php(256): XenForo_Deferred_AttachmentThumb->execute(Array, Array, 7.9999980926514, '')
#3 /var/www/vhosts/test/library/XenForo/Model/Deferred.php(390): XenForo_Model_Deferred->runDeferred(Array, 7.9999980926514, '', false)
#4 /var/www/vhosts/test/library/XenForo/Model/Deferred.php(335): XenForo_Model_Deferred->_runInternal(Array, NULL, '', false)
#5 /var/www/vhosts/test/library/XenForo/ControllerAdmin/Tools.php(159): XenForo_Model_Deferred->run(true, NULL, '', false)
#6 /var/www/vhosts/test/library/XenForo/FrontController.php(347): XenForo_ControllerAdmin_Tools->actionRunDeferred()
#7 /var/www/vhosts/test/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#8 /var/www/vhosts/test/admin.php(13): XenForo_FrontController->run()
#9 {main}
Request State
array(3) {
  ["url"] => string(51) "http://test/admin.php?tools/run-deferred"
  ["_GET"] => array(1) {
    ["tools/run-deferred"] => string(0) ""
  }
  ["_POST"] => array(5) {
    ["execute"] => string(1) "1"
    ["_xfRequestUri"] => string(24) "/admin.php?tools/rebuild"
    ["_xfNoRedirect"] => string(1) "1"
    ["_xfToken"] => string(8) "********"
    ["_xfResponseType"] => string(4) "json"
  }
}

Thanks in advance!:whistle:
 
Walter said:
There are some attachments in the database where the importer fails to set thumbnail_width and _height.
Click to expand...

Do those attachments have the values set in the vbulletin database? Did the attachment import work on the 1.2 version? I never altered anything to do with attachments in 1.4 apart from add the source table prefix.

Superfluous said:
Yeah it forces you to give those permissions before install. I said 'permissions' but I'm just assuming that from the 'failed to write' error.
Click to expand...

Check the owner/group of the files themselves.
 
Superfluous said:
Looking through the database tables I'm thinking that there were missing columns in the insert queries (assuming column names weren't defined for inserts) - for instance look at the users table where it appears avatar height/width values have found their way into the avatar_date column
Click to expand...

Same thing here. All user table fields are a mess. My own user got a 150 in the field is_banned.

The reason for this: the exporter writes the fields in a certain order and the importer does a fast LOAD DATA. I have analysed two Xenforo installs, one brand new install of 1.4 and an older one, they have different orders of the fields in the user table. The whole export/import can't work this way!
 
You must log in or register to reply here.

Similar threads

M
  • Question Question
XF 2.2 XF2 CLI import vs vBulletin Big Board Importer
Replies
1
Views
527
Slavik
Slavik
Slavik
vBulletin Big Board Importer for XenForo 2 [vBulletin 3 & vBulletin 4] [Deleted]
Replies
1
Views
810
Slavik
Slavik
Steffen
vBulletin 3/4 Moderator Log Import Command
Replies
0
Views
964
Steffen
Steffen
Kier
  • Question Question
XF 2.0 vBulletin 4 with Blog importer - known issue
Replies
8
Views
17K
Universal4
U
Mr Lucky
  • Question Question
XF 1.5 Redirect from vBulletin 3 and vBulletin 4
Replies
1
Views
489
Mr Lucky
Mr Lucky
Back
Top Bottom