vBulletin Big Board Importer [vBulletin 3 + vBulletin 4] [Paid] 1.5.0

Deriel · Aug 27, 2013

Jake Bunce said:
DP's original importer didn't support file system attachments. I added it. This is my error.

I think that the error origin is of minor importance. The fix and the honesty of the message are the important things - keep the good job

Walter · Aug 27, 2013

Jake Bunce said:
@Walter reported this and he is currently testing a fix for me. I will post it once it has been confirmed. In the meantime I wanted to alert you guys to this bug.

Thanks @Jake Bunce for providing the fix so fast, its working now.

maxicep · Aug 27, 2013

Where is the fixed file @Walter @Jake Bunce ?

Jake Bunce · Aug 28, 2013

Here is the code fix for the vB4 file system attachment bug.

Replace this code:

Code:

				$attachments = $this->db->fetchAll('
					SELECT attachment.attachmentid AS attachment_id, attachment.userid AS user_id, ' . (self::$isVB4 ? 'contentid' : 'postid') . ' AS content_id, attachment.dateline AS attachment_date, ' . (self::$isVB4 ? 'filedata.filedata' : 'attachment.filedata') . ' AS file, ' . (self::$isVB4 ? 'filedata.thumbnail_filesize' : 'attachment.thumbnail_filesize') . ', attachment.filename, ' . (self::$isVB4 ? 'filedata.extension' : 'attachment.extension') . ' AS extension
					FROM ' . $this->sourceDb . '.attachment
						' . (self::$isVB4 ? 'LEFT JOIN ' . $this->sourceDb . '.filedata ON (filedata.filedataid = attachment.filedataid)' : '') . '
					WHERE attachment.attachmentid BETWEEN ' . intval($start) . ' AND ' . intval($start + 999) . (self::$isVB4 ? ' AND attachment.contenttypeid = 1' : '') . '
					' . (self::$isVB4 ? 'GROUP BY attachment.filedataid' : '') . '
					ORDER BY attachment.attachmentid
				')
		)
		{
			echo "\r\n         " . number_format(count($attachments)) . " new attachments\r\n";

			foreach ($attachments as $attachment)
			{
			
				echo $attachment['attachment_id'] . ' ';
				

				if (self::$attachFile AND file_exists(self::$attachFile . implode('/', preg_split('//', $attachment['user_id'],  -1, PREG_SPLIT_NO_EMPTY)) . '/' . strval($attachment['attachment_id']) . '.attach'))
				{
					$attachment['file'] = file_get_contents(self::$attachFile . implode('/', preg_split('//', $attachment['user_id'],  -1, PREG_SPLIT_NO_EMPTY)) . '/' . strval($attachment['attachment_id']) . '.attach');
				}
				else
				{
					$attachment['file'] = $attachment['file'];
				}

				if (!$attachment['file'])
				{
					echo '!!MISSING DATA!!';
					continue;
				}

...with this:

Code:

				$attachments = $this->db->fetchAll('
					SELECT attachment.attachmentid AS attachment_id, attachment.userid AS user_id, ' . (self::$isVB4 ? 'contentid' : 'postid') . ' AS content_id, attachment.dateline AS attachment_date, ' . (self::$isVB4 ? 'filedata.filedata' : 'attachment.filedata') . ' AS file, ' . (self::$isVB4 ? 'filedata.thumbnail_filesize' : 'attachment.thumbnail_filesize') . ', attachment.filename, ' . (self::$isVB4 ? 'filedata.extension' : 'attachment.extension') . ' AS extension, ' . (self::$isVB4 ? 'filedata.filedataid' : 'attachment.attachmentid') . ' AS filesysid
					FROM ' . $this->sourceDb . '.attachment
						' . (self::$isVB4 ? 'LEFT JOIN ' . $this->sourceDb . '.filedata ON (filedata.filedataid = attachment.filedataid)' : '') . '
					WHERE attachment.attachmentid BETWEEN ' . intval($start) . ' AND ' . intval($start + 999) . (self::$isVB4 ? ' AND attachment.contenttypeid = 1' : '') . '
					' . (self::$isVB4 ? 'GROUP BY attachment.filedataid' : '') . '
					ORDER BY attachment.attachmentid
				')
		)
		{
			echo "\r\n         " . number_format(count($attachments)) . " new attachments\r\n";

			foreach ($attachments as $attachment)
			{
			
				echo $attachment['attachment_id'] . ' ';
				

				if (self::$attachFile AND file_exists(self::$attachFile . implode('/', preg_split('//', $attachment['user_id'],  -1, PREG_SPLIT_NO_EMPTY)) . '/' . strval($attachment['filesysid']) . '.attach'))
				{
					$attachment['file'] = file_get_contents(self::$attachFile . implode('/', preg_split('//', $attachment['user_id'],  -1, PREG_SPLIT_NO_EMPTY)) . '/' . strval($attachment['filesysid']) . '.attach');
				}
				else
				{
					$attachment['file'] = $attachment['file'];
				}

				if (!$attachment['file'])
				{
					echo '!!MISSING DATA!!';
					continue;
				}

This will be included in the next update. In the meantime you can manually apply the code fix.

Brent W · Aug 28, 2013

So I am trying to do an import for someone who purchased this and I am getting this error when running it on their database: (All of their images are stored in database if that matters).

Code:

performing avatar diff...
        24,513 users with new avatars

An exception occurred: Invalid image type given. Expects IMAGETYPE_XXX constant. in XenForo/Image/Gd.php on line 92
#0 [internal function]: XenForo_Image_Gd::createFromFileDirect('/home/test...', 6)
#1 XenForo/Image/Abstract.php(181): call_user_func(Array, '/home/test...', 6)
#2 /home/test/www/Export.php(1555): XenForo_Image_Abstract::createFromFile('/home/test...', 6)
#3 /home/test/www/Export.php(1861): DigitalPointExporter->avatars()
#4 {main}

Can't get past that. Ideas?

ehd · Aug 28, 2013

OMG, I found my mistake:

protected $sourceDb = 'DBNAME';
protected $slaveDbHost = 'localhost';
protected $sourceDBuser = '-uDBUSER';
protected $sourceDBpassword = '-pDBPASS';

Note the -u and -p I overwrote

Now it's currently running on the testserver...

Jake Bunce · Aug 28, 2013

BamaStangGuy said:
So I am trying to do an import for someone who purchased this and I am getting this error when running it on their database: (All of their images are stored in database if that matters).

Code:

performing avatar diff... 24,513 users with new avatars An exception occurred: Invalid image type given. Expects IMAGETYPE_XXX constant. in XenForo/Image/Gd.php on line 92 #0 [internal function]: XenForo_Image_Gd::createFromFileDirect('/home/test...', 6) #1 XenForo/Image/Abstract.php(181): call_user_func(Array, '/home/test...', 6) #2 /home/test/www/Export.php(1555): XenForo_Image_Abstract::createFromFile('/home/test...', 6) #3 /home/test/www/Export.php(1861): DigitalPointExporter->avatars() #4 {main}

Can't get past that. Ideas?

I got that error with bmp images so I excluded that type:

Code:

				if ($attachment['thumbnail_filesize'] > 0 AND $attachment['extension'] != 'bmp')

You need to debug the image type and probably exclude it from the thumbnailing.

There should probably be a white list of image types for this.

Brent W · Aug 28, 2013

What is the best method to debug this? How do I find out which record is throwing the error?

Brent W · Aug 29, 2013

What are all the valid image types that xenForo accepts? Would probably be easier to just accept gif, jpg, jpeg, png right? I assume that is what you mean by a white list.

Slavik · Aug 29, 2013

BamaStangGuy said:
What is the best method to debug this? How do I find out which record is throwing the error?

Check the XenForo server error log.

Brent W · Aug 29, 2013

I have 214 pages worth...

A pile of these:

Code:

Server Error Log
Error Info
ErrorException: Undefined offset: 10 - Export.php:370
Generated By: Unknown Account, 52 minutes ago
Stack Trace
#0 /home/test/www/Export.php(370): XenForo_Application::handlePhpError(8, 'Undefined offse...', '/home/test...', 370, Array)
#1 /home/test/www/Export.php(1861): DigitalPointExporter->users()
#2 {main}
Request State
array(3) {
  ["url"] => string(7) "http://"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}

Brent W · Aug 29, 2013

That probably has to do with the usergroupids not matching. Looking into that.

Slavik · Aug 29, 2013

BamaStangGuy said:
That probably has to do with the usergroupids not matching. Looking into that.

Yup

Brent W · Aug 29, 2013

Code:

Request State
array(3) {
  ["url"] => string(7) "http://"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}

Same error as before expect this is new (image error)

Slavik · Aug 29, 2013

Off the top of my head, you should be able to add bmp to the XenForo allowed filetypes in the options menu, I think.

As for finding the one erroring out, is the importer not displaying the avatar progress and erroring immediately? If so, its probably a good bet it one of the first users.

Brent W · Aug 29, 2013

Code:

exporting private messages...
          [forking process for conversation master]
          conversation message (264,730 records in 8.38s)...
          conversation recipient (453,006 records in 3.65s)...
          conversation user (453,006 records in 4.54s)...
          skipping private conversation IPs
Total time for private messages: 16.59s

performing avatar diff...
        24,513 users with new avatars

An exception occurred: Invalid image type given. Expects IMAGETYPE_XXX constant. in XenForo/Image/Gd.php on line 92
#0 [internal function]: XenForo_Image_Gd::createFromFileDirect('/home/test...', 6)
#1 XenForo/Image/Abstract.php(181): call_user_func(Array, '/home/test...', 6)
#2 /home/test/www/Export.php(1551): XenForo_Image_Abstract::createFromFile('/home/test...', 6)
#3 /home/test/www/Export.php(1857): DigitalPointExporter->avatars()
#4 {main}

That is a bit before and the last line is it before quitting

Jake Bunce · Aug 29, 2013

BamaStangGuy said:
What is the best method to debug this? How do I find out which record is throwing the error?

Oh it's during the avatar export. I thought it was attachments. Debugging still applies though. Uncomment this echo statement in the avatar step:

Code:

				// echo $user['userid'] . ' ';

Then run the export again. When the error happens, look to the last userid before the error. That will be the problem avatar.

Brent W · Aug 29, 2013

Thanks, I have the ID now. 352

Is there a query I can run to delete that users avatar in the vBulletin database?

Brent W · Aug 29, 2013

Can I just delete the whole row in customavatars for that userid?

Jake Bunce · Aug 29, 2013

BamaStangGuy said:
Thanks, I have the ID now. 352

Is there a query I can run to delete that users avatar in the vBulletin database?

Edit that user in the vB Admin CP. The edit page has an option to edit their avatar.

vBulletin Big Board Importer [vBulletin 3 + vBulletin 4] [Paid] 1.5.0

Well-known member

Well-known member

Active member

Well-known member

Well-known member

Well-known member

Well-known member

Well-known member

Well-known member

XenForo moderator

Well-known member

Well-known member

XenForo moderator

Well-known member

XenForo moderator

Well-known member

Well-known member

Well-known member

Well-known member

Well-known member

Similar threads

We value your privacy