XF 2.0 Import addon (1.0.1a) does not ask for attachments path (vBulletin 4.2 to xenforo 2)

[electric.larry]

I am Using XFI 1.0.1a (including this patch by Jake Bunce) to import vBulletin 4.2 (attachments saved in filesystem) to xenForo 2. Import takes around 6 hours when run from the CLI. Almost half the time compared to an import started within a browser. At the end, most of the imported content looks fine (except some permissions), only the attachments are missing.

The vBulletin Import notes (https://xenforo.com/xf2-docs/manual/import-notes-vbulletin/) state, that the import script will ask for the system path where attachments and avatars are stored, when it detects, that attachments are not stored in the vBulletin DB. I performed a fresh import twice but was never asked for the attachments path.

I am performing the import on another host where the attachments are in a different directory than on the live system. Is it possible, that the import script reads the attachments directory from the vB database and does not issue a warning/error, if that directory can not be found?

Would I solve this issue by moving the attachments to the same directory as on the live system?

Thanks!

 

[Chris D]

If you select the attachments step when you set up the importer then it should ask you for the path.

It’s doing it for me on my test imports.

 

[electric.larry]

Hi Chris,

thanks for your quick reply.

Meanwhile I tried the import four times with XFI 1.0.1 and XFI 1.0.0 based on a fresh xenForo install. I tried with Jake Bunce's patch and without it. I never get asked to enter the attachments path. The vBulletin Board is definitly set to store attachments in the filesystem.

I am performing the import on another host inside a docker container (if that makes any difference). I only copied the vBulletin database and the attachments directory to this host. I placed the attachments directory in the same directory where it is found on the live system and double checked, that the attachments dir is accessible. Does the importer require to copy any other vBulletin files to the system where the import is performed? If yes, do I have to place the files exactly in the same directories as on the live system?

Could you please point out, where the importer decides wether it's dealing with attachments in the filesystem or in the DB? Maybe I can find out, which settings of our vBulletin cause this problem.

Should the import script ask where attachments are stored BEFORE the import starts or only when the import is finalized?

Here is what I see, when starting the import:

 

[Slavik]

I have bumped into this once or twice where vb for some really weird and stupid reason ended up splitting attachments between both the database and filesystem. I assume that at some point an attachment migration from one to the other has been run and been interrupted along the way.

I also presume that some flags are changed in the database (or not changed) and that is why the attachment importer skips them silently.

Ultimately, as annoying as it is, having it re-run the move process from one to the other usually fixes the issue.

 

[electric.larry]

Thanks a lot, I'll try to find the place in the import script, where it makes the check. If that seems too complicated, I'll try to move them back into the DB.

 

[electric.larry]

@Slavik's hint lead me to the root of my import issues. Some missing values in vBulletin's setting table caused the import script to assume it's dealing with attachments in the database and so it never asked for the file system path where attachments are stored.

I tracked down the place, where the import script decides wether attachments are stored in file system or not. It reads attachpath and attachfile values from the vBulletin setting table. It can be found in src/addons/XFI/Import/Importer/vBulletin.php around line 153:

if (!empty($options['attachfile']) && !empty($options['attachpath']))
{
   $baseConfig['attachpath'] = trim($options['attachpath']);
}

A few lines above those settings are read from vBulletin's setting table:

if ($validDbConnection)
{
    try
    {
        $options = $sourceDb->fetchPairs("
            SELECT varname, value
            FROM " . $fullConfig['db']['prefix'] . "setting
            WHERE varname IN('attachpath', 'attachfile', 'avatarpath', 'usefileavatar', 'languageid', 'bburl', 'blogattachfile', 'blogattachpath')
        ");
}

I inspected the setting table of vBulletin and realized it was empty due to an error while importing the vBulletin database into the database of the new xF server:

ERROR 1265 (01000) at line xxx: Data truncated for column 'method' at row 1

Turns out the new mysql server was set to strict mode, while the original vBulletin database had strict mode disabled. Disabling strict mode allowed a full import of the original vBulletin database.

SHOW VARIABLES LIKE 'SQL_MODE';
+---------------+-------------------------------------------------------------------------------------------+
| Variable_name | Value                                                                                     |
+---------------+-------------------------------------------------------------------------------------------+
| sql_mode      | STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION |
+---------------+-------------------------------------------------------------------------------------------+

SET GLOBAL sql_mode = 'ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';

So finally, after making sure that attachpath and attachfile variables are found in vBulletin's setting table, the installer requires me to input the attachments path while setting up the import.



In around six hours I'll hopefully have a fully working xF board

Thanks a lot for your great support!