Fixed xfmg 1.1.0 - Error when importing tags from PhotoPost

[Triops]

This is just a new test, after it worked in earlier xfmg versions.

* Do a vB 3.7 import into fresh xf installation 1.5.0 + xfmg 1.1.0, no problem, finish import, rebuild search index
* Do a PP import, all steps run smooth till importing tags
* tag import starts, but seconds after start I get this error:

Error Info
Zend_Db_Statement_Mysqli_Exception: Mysqli statement execute error : Duplicate entry 'xengallery_media-3017-244' for key 'content_type_id_tag' - library\Zend\Db\Statement\Mysqli.php:214
Generated By: XenM, Today at 3:43 PM
Stack Trace
#0 E:\xampp\htdocs\xenforo\library\Zend\Db\Statement.php(297): Zend_Db_Statement_Mysqli->_execute(Array)
#1 E:\xampp\htdocs\xenforo\library\Zend\Db\Adapter\Abstract.php(479): Zend_Db_Statement->execute(Array)
#2 E:\xampp\htdocs\xenforo\library\Zend\Db\Adapter\Abstract.php(574): Zend_Db_Adapter_Abstract->query('INSERT INTO `xf...', Array)
#3 E:\xampp\htdocs\xenforo\library\XenForo\Model\Import.php(1657): Zend_Db_Adapter_Abstract->insert('xf_tag_content', Array)
#4 E:\xampp\htdocs\xenforo\library\XenGallery\Model\Importers.php(476): XenForo_Model_Import->importTag('Diamond', 'xengallery_medi...', '3017', Array)
#5 E:\xampp\htdocs\xenforo\library\XenGallery\Importer\PhotopostVb.php(922): XenGallery_Model_Importers->importTag('Diamond', 'xengallery_medi...', '3017', Array)
#6 E:\xampp\htdocs\xenforo\library\XenGallery\Importer\PhotopostVb.php(891): XenGallery_Importer_PhotopostVb->_importTag(Array, Array)
#7 E:\xampp\htdocs\xenforo\library\XenForo\Importer\Abstract.php(97): XenGallery_Importer_PhotopostVb->stepContentTags(3004, Array)
#8 E:\xampp\htdocs\xenforo\library\XenForo\ControllerAdmin\Import.php(187): XenForo_Importer_Abstract->runStep(Object(XenForo_ControllerAdmin_Import), Object(XenForo_ImportSession), 'contenttags', 3004, Array)
#9 E:\xampp\htdocs\xenforo\library\XenForo\ControllerAdmin\Import.php(135): XenForo_ControllerAdmin_Import->_runStep(Object(XenGallery_Importer_PhotopostVb), Object(XenForo_ImportSession), 'contenttags', 3004, Array)
#10 E:\xampp\htdocs\xenforo\library\XenForo\FrontController.php(347): XenForo_ControllerAdmin_Import->actionImport()
#11 E:\xampp\htdocs\xenforo\library\XenForo\FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#12 E:\xampp\htdocs\xenforo\admin.php(13): XenForo_FrontController->run()
#13 {main}
Request State
array(3) {
  ["url"] => string(51) "http://localhost:81/xenforo/admin.php?import/import"
  ["_GET"] => array(1) {
    ["import/import"] => string(0) ""
  }
  ["_POST"] => array(1) {
    ["_xfToken"] => string(8) "********"
  }
}

Copied from Server Error Log.
This was:

Mysqli statement execute error : Duplicate entry 'xengallery_media-3017-244' for key 'content_type_id_tag'
Today at 3:43 PM - library\Zend\Db\Statement\Mysqli.php:214

Trying to continue import immediatly throws the next error for another entry

Mysqli statement execute error : Duplicate entry 'xengallery_media-3006-292' for key 'content_type_id_tag'
5 minutes ago - library\Zend\Db\Statement\Mysqli.php:214

Import will not go further.

 

[Triops]

Ha, just after writing this thread I had an idea:
I have no add-ons installed, but XenTag 2.1.0. Nothing else. Could it be because of that? I want to use it till there is an importer for core tags from vB 3.7. But XenTag does not care for PhotoPost tags, should not conflict.

 

[Chris D]

XenTag shouldn't be involved here, no.

 

[Chris D]

This is fixed, now. I tracked this down.

It's basically because of how PhotoPost store their tags. They store them as a string against each photo. Tags can be separated by commas or spaces, so given a string like:

"hand, thumbs, thumbs down"

That actually translates to having 4 tags named:

hand, thumbs, thumbs, down

And that's where the duplicate comes in.

To fix this, it's relatively simple if you wanted to try it yourself.

In the file, library/XenGallery/Importer/PhotopostVb.php and/or library/XenGallery/Importer/PhotopostXf.php, find:

$keywords = preg_split('/[\ \n\,]+/', $tag['keywords']);

And add directly below:

$keywords = array_unique(array_map('utf8_strtolower', $keywords));

In addition to this, find:

$importedTagId = $model->importTag($tag['keyword'], 'xengallery_media', $mediaId, array(

And change to:

$importedTagId = $model->importTag($this->_convertToUtf8($tag['keyword'], true), 'xengallery_media', $mediaId, array(

That should sort the problem out for you. It's possible you may need to start the import again, or, if possible, start the content tag import again after deleting any tags it may have already imported.

 

[Chris D]

The above fix has been revised, slightly @DragonFlames and @PyroM

In addition to this, find:

$importedTagId = $model->importTag($tag['keyword'], 'xengallery_media', $mediaId, array(

And change to:

$importedTagId = $model->importTag($this->_convertToUtf8($tag['keyword'], true), 'xengallery_media', $mediaId, array(

 

[DragonFlames]

Thanks for the note. I think I have to start again from scratch with a fresh install as I do not know how to stop the import and delete all the imported tags and then start it at the point of importing all tags from photo 1. It took some days to get this far so I won't be back to this point for some time yet.

I guess the only file I need to edit for XF+Photopost to XF+XFMG is: PhotopostXf.php

I found the line in there and added the new one below it, plus now done the second change too.

Steve

 

[Triops]

You can empty tables xf_tag and xf_tag_content with phpmyadmin. Just empty them, do not delete.
When reloading importer, it should start with importing content tags from scratch. Continue with importer and try to get to a successful end. After that, when you are sure, its probably better to restart PhotoPost import totally with a fresh backup, yes. At least that's the way I'll do it.

Thank you @Chris D for your great help!