Resource icon

vBulletin Big Board Importer [vBulletin 3 + vBulletin 4] [Paid] 1.5.0

No permission to buy ($150.00)
Hi,

I have a general issue / question on the Big Board Importer script.

The Export.php not only uses the source database but also the target database ($this->db).
What I don't understand is this for example:

PHP:
                $users = $this->db->fetchAll('
                    SELECT user.*, COALESCE(IF(customavatar.dateline, customavatar.dateline, customprofilepic.dateline), 0) AS avatar_date, COALESCE(IF(customavatar.dateline, customavatar.width, customprofilepic.width), 0) AS avatar_width, COALESCE(IF(customavatar.dateline, customavatar.height, customprofilepic.height), 0) AS avatar_height, LOWER(SUBSTRING(PASSWORD(MD5(CONCAT(user.salt, user.userid))), 2)) AS remember_key,
                        (SELECT COUNT(*) FROM ' . $this->sourceDb . '.' . self::$tablePrefix . 'reputation as reputation INNER JOIN ' . $this->sourceDb . '.' . self::$tablePrefix . 'post as post ON (post.postid = reputation.postid) INNER JOIN ' . $this->sourceDb . '.' . self::$tablePrefix . 'thread as thread ON (thread.threadid = post.threadid) WHERE post.visible = 1 AND thread.visible = 1 AND reputation.userid = user.userid AND (reputation > 0 OR (reason = \'\' AND reputation = 0))) AS like_count
                    FROM ' . $this->sourceDb . '.' . self::$tablePrefix . 'user as user
                        LEFT JOIN ' . $this->sourceDb . '.' . self::$tablePrefix . 'customavatar as customavatar ON (customavatar.userid = user.userid)
                        LEFT JOIN ' . $this->sourceDb . '.' . self::$tablePrefix . 'customprofilepic as customprofilepic ON (customprofilepic.userid = user.userid)
                    ORDER BY user.userid
                    LIMIT ' . ($limit * 10000) .', 10000
                ')
                )

This part of the code tries to access a table customprofile in the xenforo database which of course never exists. How is this supposed to even work?

Furthermore: The script assumes that the target database and the source database both have the same table prefix which also seems to be a bug. The user table in vBulletin is "user" in XenForo it's "xf_user".

Has anyone experienced the same issues? Or am I just misunderstanding the script?

Thanks a lot for hints and comments.

-Fabian-

It's using the XF database connection to access the vB database. The rendered query names the vB database for all of those tables, so it looks something like this:

Code:
SELECT *
FROM vbulletin.user AS user
...

The prefix is only used for vB tables, not XF tables.

The only real possible issue here is that both the vB and XF databases need to be accessible by the same MySQL user to allow for the vB database to be accessed using the XF db connection.
 
Hello we've bought this importer and its working fine. But we need a bill for this extension, where can I download or get one? I found nothing at my accounts.

Thanks
 
Hello we've bought this importer and its working fine. But we need a bill for this extension, where can I download or get one? I found nothing at my accounts.

Thanks

It would simply be your paypal receipt.
 
I encountered the below error upon attempting to export posts. Any idea how to fix it?
Code:
          postssh: -c: line 0: syntax error near unexpected token `)'
sh: -c: line 0: `mysql -hlocalhost o_forum -uo_dbuser -p)yB0OSIDYxnt -N -q -e "SET NAMES binary;SET @position = -1, @previous = 0, @last_state = 0, @ip_id = 0;SELECT SQL_NO_CACHE post.postid AS post_id, post.threadid AS thread_id, post.userid AS user_id, post.username, post.dateline AS post_date, post.pagetext AS message, @ip_id:=@ip_id+1 AS ip_id, IF(post.visible = 0, 'moderated', IF(post.visible = 2, 'deleted', 'visible')) AS message_state, attach AS attach_count, IF(@previous = post.threadid, @position:=@position+@last_state, @position:=(0 * @previous:=post.threadid)) + ((@last_state:=IF(post.visible = 1, 1, 0)) * 0) AS position, 0 AS likes, 'a:0:{}' AS like_users, COALESCE((SELECT infractionid FROM infraction as infraction WHERE infraction.postid = post.postid AND action IN(0, 1) LIMIT 1), 0) AS warning_id, '' AS warning_message, 0 AS last_edit_date, 0 AS last_edit_user_id, 0 AS edit_count FROM post as post WHERE 1 ORDER BY post.threadid, post.postid" | sed -r 's/\[quote="?([^]";]*?);([^]";]*?)"?\]/[quote="\1, post: \2"]/ig;s/\[highlight\]//ig;s/\[\/highlight\]//ig;s/\[nofollow/\[url/ig;s/\[\/nofollow/\[\/url\]/ig;s/\[noparse\]/\[plain\]/ig;s/\[\/noparse\]/\[\/plain\]/ig;s/\[strike\]/\[s\]/ig;s/\[\/strike\]/\[\/s\]/ig;s/\[video\]http\:\/\/www.youtube.*v=([^&[]*?).*\[\/video\]/\[media=youtube\]\1\[\/media\]/ig' | iconv -f LATIN1 -t UTF8 > /home/e/public_html/import_data/xf_post.txt'
 
Last edited:
Ah, nevermind, I got it. I have to quote any part of the command that has a parentheses in it, so -p")yB0OSIDYxnt". No, that's not the real password.
 
Last edited:
I get the "XenForo_Exception: Invalid image type given. Expects IMAGETYPE_XXX constant" error when I export avatars. I have commented in the
Code:
                echo $user['userid'] . ' ';
line to find the offending user IDs. Is there a better way to find all users with bmp avatars other than to run the avatar exporter over and over again? Each time I run it it takes 10 minutes or so. I'm on user 10...
 
I get the "XenForo_Exception: Invalid image type given. Expects IMAGETYPE_XXX constant" error when I export avatars. I have commented in the
Code:
                echo $user['userid'] . ' ';
line to find the offending user IDs. Is there a better way to find all users with bmp avatars other than to run the avatar exporter over and over again? Each time I run it it takes 10 minutes or so. I'm on user 10...
On a related note, in the db you're importing from, make sure to create a test thread with special characters (such as 'ɟpsɐ') in the title of the thread, in the post of the thread, in the user title (for a test user you create on the old board), and any other places you'd like to be sure convert properly with special chars.

We did this and found some places (the titles and other spots) weren't converting extended/Unicode characters properly with the script as is, so we had to do a few edits to the script to make sure it was handling cases like this.
 
Thanks for that tip. Will you post your edits for converting extended/Unicode characters here for us? Or perhaps send me your modified file? I will then do a file diff to determine the edits.

On the topic of finding bmp avatars, I found mine by searching for %.bmp% in the vB customavatar table. From there you can simply delete them if you so choose.
 
I got to the import stage but got a lot of warnings and a couple errors, which I have attached. A migration that I completed with the standard importer is working, but the site that this importer provided does not display the forums. Must any of these errors in the attached log be fixed?

It also appears that I am the only moderator and administrator, but on the standard migration I correctly see the others. It appears that the permissions may be incorrect. What should I do to fix these?
 

Attachments

Last edited:
Does this importer import Custom User Fields? I see that mine were not imported.

Also, I get this error when rebuilding caches:
Code:
Server Error Log
Error Info
Zend_Db_Statement_Mysqli_Exception: Mysqli statement execute error : Incorrect integer value: '' for column 'user_group_id' at row 2 - library/Zend/Db/Statement/Mysqli.php:214
Generated By: admin, Today at 11:24 AM
Stack Trace
#0 /home/e/public_html/library/Zend/Db/Statement.php(297): Zend_Db_Statement_Mysqli->_execute(Array)
#1 /home/e/public_html/library/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
#2 /home/e/public_html/library/XenForo/DataWriter/User.php(1111): Zend_Db_Adapter_Abstract->query('\n\t\t\tINSERT INTO...')
#3 /home/e/public_html/library/XenForo/Deferred/User.php(38): XenForo_DataWriter_User->rebuildUserGroupRelations()
#4 /home/e/public_html/library/XenForo/Model/Deferred.php(295): XenForo_Deferred_User->execute(Array, Array, 7.99999785423, '')
#5 /home/e/public_html/library/XenForo/Model/Deferred.php(429): XenForo_Model_Deferred->runDeferred(Array, 7.99999785423, '', false)
#6 /home/e/public_html/library/XenForo/Model/Deferred.php(374): XenForo_Model_Deferred->_runInternal(Array, NULL, '', false)
#7 /home/e/public_html/library/XenForo/ControllerAdmin/Tools.php(159): XenForo_Model_Deferred->run(true, NULL, '', false)
#8 /home/e/public_html/library/XenForo/FrontController.php(351): XenForo_ControllerAdmin_Tools->actionRunDeferred()
#9 /home/e/public_html/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#10 /home/e/public_html/admin.php(13): XenForo_FrontController->run()
#11 {main}
Request State
array(3) {
  ["url"] => string(55) "http://example.com/admin.php?tools/run-deferred"
  ["_GET"] => array(1) {
    ["tools/run-deferred"] => string(0) ""
  }
  ["_POST"] => array(5) {
    ["execute"] => string(1) "1"
    ["_xfRequestUri"] => string(34) "/admin.php?tools/rebuild&success=1"
    ["_xfNoRedirect"] => string(1) "1"
    ["_xfToken"] => string(8) "********"
    ["_xfResponseType"] => string(4) "json"
  }
}
 
Last edited:
Must non-Xenforo-default user groups that exist in the source vBulletin forum be created in Xenforo before import, or does the importer create user groups as the standard importer does?
 
Must non-Xenforo-default user groups that exist in the source vBulletin forum be created in Xenforo before import, or does the importer create user groups as the standard importer does?

You need to manually create the groups in XenForo. Then edit the top of the Export file to create your group id maps.
 
Does this importer import Custom User Fields? I see that mine were not imported.

Also, I get this error when rebuilding caches:
Code:
Server Error Log
Error Info
Zend_Db_Statement_Mysqli_Exception: Mysqli statement execute error : Incorrect integer value: '' for column 'user_group_id' at row 2 - library/Zend/Db/Statement/Mysqli.php:214
Generated By: admin, Today at 11:24 AM
Stack Trace
#0 /home/e/public_html/library/Zend/Db/Statement.php(297): Zend_Db_Statement_Mysqli->_execute(Array)
#1 /home/e/public_html/library/Zend/Db/Adapter/Abstract.php(479): Zend_Db_Statement->execute(Array)
#2 /home/e/public_html/library/XenForo/DataWriter/User.php(1111): Zend_Db_Adapter_Abstract->query('\n\t\t\tINSERT INTO...')
#3 /home/e/public_html/library/XenForo/Deferred/User.php(38): XenForo_DataWriter_User->rebuildUserGroupRelations()
#4 /home/e/public_html/library/XenForo/Model/Deferred.php(295): XenForo_Deferred_User->execute(Array, Array, 7.99999785423, '')
#5 /home/e/public_html/library/XenForo/Model/Deferred.php(429): XenForo_Model_Deferred->runDeferred(Array, 7.99999785423, '', false)
#6 /home/e/public_html/library/XenForo/Model/Deferred.php(374): XenForo_Model_Deferred->_runInternal(Array, NULL, '', false)
#7 /home/e/public_html/library/XenForo/ControllerAdmin/Tools.php(159): XenForo_Model_Deferred->run(true, NULL, '', false)
#8 /home/e/public_html/library/XenForo/FrontController.php(351): XenForo_ControllerAdmin_Tools->actionRunDeferred()
#9 /home/e/public_html/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#10 /home/e/public_html/admin.php(13): XenForo_FrontController->run()
#11 {main}
Request State
array(3) {
  ["url"] => string(55) "http://example.com/admin.php?tools/run-deferred"
  ["_GET"] => array(1) {
    ["tools/run-deferred"] => string(0) ""
  }
  ["_POST"] => array(5) {
    ["execute"] => string(1) "1"
    ["_xfRequestUri"] => string(34) "/admin.php?tools/rebuild&success=1"
    ["_xfNoRedirect"] => string(1) "1"
    ["_xfToken"] => string(8) "********"
    ["_xfResponseType"] => string(4) "json"
  }
}

Custom fields are not imported by default. You need to manually configure that section of the Export file. Search for "xf_user_field". That should find it.

That error likely indicates that your xf_user columns are misaligned. Column order is very important with this importer. What version of XF are you running? This importer was last aligned to XF 1.4. It is my intention to eventually rewrite this importer so that it automatically handles the column ordering.
 
I got to the import stage but got a lot of warnings and a couple errors, which I have attached. A migration that I completed with the standard importer is working, but the site that this importer provided does not display the forums. Must any of these errors in the attached log be fixed?

It also appears that I am the only moderator and administrator, but on the standard migration I correctly see the others. It appears that the permissions may be incorrect. What should I do to fix these?

You can actually ignore all of those errors. I looked over your log and those are normal. This importer doesn't do strict data validation or complete inserts which means you will get some harmless errors like you did. These can be ignored. If I get around to rewriting this script then it will do a better job of avoiding these harmless errors.

After the Export + Import these are a few gotchas:

1) User_id 1 usually needs to be manually promoted to admin. This means editing that record in xf_user and setting is_admin to 1. That should get you into the Admin CP.

2) Yes, permissions need to be rebuilt after the Import.

Admin CP -> Tools -> Rebuild Caches
> Rebuild User Information
> Rebuild Forum Information


These rebuilds should fix permissions and make it so the forums show correctly.

3) Note my previous reply... it appears your user records might be suffering from column misalignment which is something that would have to be fixed during the Import process before any rebuilds.
 
Custom fields are not imported by default. You need to manually configure that section of the Export file. Search for "xf_user_field". That should find it.

That error likely indicates that your xf_user columns are misaligned. Column order is very important with this importer. What version of XF are you running? This importer was last aligned to XF 1.4. It is my intention to eventually rewrite this importer so that it automatically handles the column ordering.
I found xf_user_field in Export.php. I installed the latest version, 1.5.1. Could you grab code from the standard importer to handle the automatic ordering?
 
I found xf_user_field in Export.php. I installed the latest version, 1.5.1. Could you grab code from the standard importer to handle the automatic ordering?

Actually I looked at the upgrade files and XF 1.5 should be fine with this importer. I don't see any changes that would break things. Yet your user records appear to have problems. Search for this code:

Code:
fwrite($fp_user, str_replace('\\', '\\\\', "$user[userid]	" . html_entity_decode($user['username']) . "	$user[email]		" . ($user['customtitle'] ? html_entity_decode($user['usertitle']) : '') . "	1	0	" . $importModel->resolveTimeZoneOffset($user['timezoneoffset'], $user['options'] & 64) . "	1	1	$user[usergroupid]	$user[membergroupids]	$userGroupDisplay	$permissionComboId	$user[posts]	$user[pmunread]	$user[joindate]	$user[lastactivity]	0	0	$user[avatar_date]	192	192		" . ($user['usergroupid'] == 1 ? 'email_confirm' : 'valid') . "	" . ($user['usergroupid'] == 4 ? '1' : '0') . "	" . ($user['usergroupid'] == 3 ? '1' : '0') . "	" . ($is_banned ? 1 : 0) . "	$user[like_count]	$user[ipoints]	" . (($user['usergroupid'] == 4 OR $user['usergroupid'] == 3) ? '1' : '0')) . "\n");

That is the code which writes the user records to the intermediate data files. Basically you need to compare this ordering to the ordering that exists in your xf_user table in your XF database. The fields are tab-delimited. This was last aligned for XF 1.4 I believe. Maybe your XF forum has some addons which changed the ordering.

(this obviously requires an ability to read PHP strings)
 
Could you grab code from the standard importer to handle the automatic ordering?

Nope. It's not a matter of grabbing some code from the official importer. The two importers handle the data differently. This big board importer uses data files to get the fastest possible performance, and those files are order-sensitive by nature.

If I get around to rewriting this importer then I would implement some system in the Export to automatically detect your column orders in the XF tables and then use those orders when writing out the data files. This as opposed to having the orders hard-coded which is the current situation. This is not a minor change and it would require a complete rewrite.
 
2) ...
These rebuilds should fix permissions and make it so the forums show correctly.

3) Note my previous reply... it appears your user records might be suffering from column misalignment which is something that would have to be fixed during the Import process before any rebuilds.
2) OK, strangely I can see the forums now. Maybe it was a caching issue.

3) I don't quite understand this part. Are you saying that fixing the xf_user_field stuff in Export.php may cause the administrators and moderators and their moderated forums to be imported correctly. Here is my user group mapping:
Code:
// Created in the destination Xenforo
// 5-Supermods
// 6-Banned
// 7-Active
// 8-PMs
// 9-Summarizers
    // MAP VB GROUP IDS TO XF GROUP IDS
    public static $groupMap = array(
                1 => 1,
                2 => 2,
                3 => 1,
                4 => 1,
                5 => 5,
                6 => 3,
                7 => 4,
                8 => 6,
                18 => 7,
                21 => 8,
                23 => 9,
            );
 
Top Bottom