RM 1.1 Import Files from DownloadsII

[Sadik B]

This is more of a discussion thread and not a support question. My Oracle Forums has about 12000 files in DownloadsII. I have been considering moving this site to XF for over a year now.

Have any of you successfully imported from Downloads II several thousand files? Is there an importer I may have overlooked?

If not, I am considering coding an importer for my needs. Would there be any commercial interest in it? Else I could just import by directly running queries on the database and not bother with an importer. Thoughts?

 

[0ptima]

LDM has comments with and without ratings. Is this the same for Downloads II ?

Yes

LDM also has automatic forum thread creation, which is basically a thread tied to the resource. Some resources have it, while others do not. Comments are replicated in the thread.Does downloads II have something similar?

No

Any idea if this can be imported through downloads II into RM?

To add comments to the RM, you need to include a rating between 1-5. Therefore comments without ratings should be added to the thread that is associated with resource.

 

[Alpha1]

It seems to me I will need a LDM -> RM importer after all.

Part of my files are video's anyway, which need to go into the MG.

 

[Sadiq6210]

I included the code to update the "latest version release date"

        $db->update('xf_resource_version', array(
            'release_date' => $dl['dateadded'],
        ), 'resource_version_id = ' . $version['resource_version_id']);

Also I updated few codes to avoid the errors if vBulletin database not using UTF-8
The new file is in the attachments.

After complete the importing, we need to run 2 queries in database:

update xf_resource_version
set version_string = DATE_FORMAT(FROM_UNIXTIME(release_date),'%Y-%m-%d')

&

update xf_resource_rating
set version_string = DATE_FORMAT(FROM_UNIXTIME(rating_date),'%Y-%m-%d')

@0ptima ,
About the file ID, even @Sadik B file has the same problem.
But the problem does not happen in "categories importing", it happen only with (Files importing)
Categories imported with identical ID (example: 1,3,5,10,17,....) as per the ID in vBulletin database, while the files imported in order (1,2,3,4,....) even if it is different in vBulletin database.

Are you using the same method to import the categories and files?

 

[0ptima]

But the problem does not happen in "categories importing", it happen only with (Files importing)
Categories imported with identical ID (example: 1,3,5,10,17,....) as per the ID in vBulletin database, while the files imported in order (1,2,3,4,....) even if it is different in vBulletin database.

Im starting to think its not even possible based on what I have seen in the resource.php data writer




i think at this point it would be best to create a redirect script.

 

[Sadiq6210]

Im starting to think its not even possible based on what I have seen in the resource.php data writer

View attachment 113210

i think at this point it would be best to create a redirect script.

I think I will add the missing rows in vBulletin database then I will import it.
If there is no missing ID, then all files will be imported with same ID

The problem is: I have 152 missing ID, so I need to create 152 rows manually.

 

[Sadik B]

Ah Sadiq... Sorry I was too busy to respond. Doesn't help if clients email you at 1AM!

Yes you are right you are not seeing the right IDs and that is not because of the deleted files in vbulletin, it's because of the lazy approach I took to make sure Ids carried over.

For now, do this

Open the thread DataWriter in xenforo

library/XenForo/DataWriter/Discussion/Thread.php

On line 175 comment out this line

//$this->set('prefix_id', 0); // invalid prefix

Basically the "right" approach would have been to extend the thread datawriter with an addon and creating a new column in the thread table to use to "pass" the resource id from file id. But I took the shortcut of passing the id through the thread prefix column (Since I am deleting the threads, it does not matter!). That is why you are seeing wrong ids because the prefix check in the datawriter is silently failing. (if you are using my importer).

Comment out the above line and try again.

Regards,
- Sadik

 

[Sadiq6210]

Ah Sadiq... Sorry I was too busy to respond. Doesn't help if clients email you at 1AM!

Yes you are right you are not seeing the right IDs and that is not because of the deleted files in vbulletin, it's because of the lazy approach I took to make sure Ids carried over.

For now, do this

Open the thread DataWriter in xenforo

library/XenForo/DataWriter/Discussion/Thread.php

On line 175 comment out this line

//$this->set('prefix_id', 0); // invalid prefix

Basically the "right" approach would have been to extend the thread datawriter with an addon and creating a new column in the thread table to use to "pass" the resource id from file id. But I took the shortcut of passing the id through the thread prefix column (Since I am deleting the threads, it does not matter!). That is why you are seeing wrong ids because the prefix check in the datawriter is silently failing. (if you are using my importer).

Comment out the above line and try again.

Regards,
- Sadik

It is working perfect , you save my time
I think at this point we have a complete importer DownloadsII to Resources

Special thanks for you @0ptima and @Sadik B

_________________________

BTW Sadik, don't forget to run this query to update the version date of the comments. In you website it is all 14/7/2015

update xf_resource_rating
set version_string = DATE_FORMAT(FROM_UNIXTIME(rating_date),'%Y-%m-%d')

 

[0ptima]

Great collaboration guys! Ill package this up and release it as an addon.

@sadiq6210 Is that query really required?

@Sadik B How did you setup the redirects?

 

[Sadiq6210]

Great news to know that you will release it as an addon

@sadiq6210 Is that query really required?

Yes, this will update the comments date, see this screen-shot from Sadik forum



The query will update the version date to be same comment date.

You could build on this copy:
RM 1.1 - Import Files from DownloadsII

 

[0ptima]

I cleaned up the addon and added it to the RM
Beta - DownloadsII -> XenForo Resource Manager Importer

I changed the code so commenting this //$this->set('prefix_id', 0); is no longer required and file ids will be kept.

@sadiq6210 Can you uninstall the VB Pro Garage imported and instal the downloadsII importer from the RM and test it out? Thanks.

 

[Sadik B]

BTW Sadik, don't forget to run this query to update the version date of the comments. In you website it is all 14/7/2015

update xf_resource_rating
set version_string = DATE_FORMAT(FROM_UNIXTIME(rating_date),'%Y-%m-%d')

Did I tell you I am lazy?!

 

[adamsmasher]

This worked great for me except for one problem... download links don't work.

All of the information about a download from Downloads II is present, name, image, etc, but the file itself doesn't seem to be linked to it properly.



Not sure if I missed a step.

 

[adamsmasher]

This worked great for me except for one problem... download links don't work.

All of the information about a download from Downloads II is present, name, image, etc, but the file itself doesn't seem to be linked to it properly.

View attachment 117079

Not sure if I missed a step.

Figured out what was up. I noticed that some files were actually fine, and it turns out it's because my test importing was done with an older vBulletin db copy. Between when I made the copy and now we updated the files used by some downloads, so the files in the directory didn't match what the old db copy had. Those that were still the same worked as expected. Mystery solved!

 

[0ptima]

@adamsmasher

Glad to hear that it worked. Be sure to use this version.
https://xenforo.com/community/resources/downloadsii-xenforo-resource-manager-importer.4465/

 

[Scandal]

Is there any importer from DownloadsII (vBulletin 3.8) to XFRM latest version?

 

[ChicagoCoin]

I recently Imported VBulletin 4.2.5 into a new install of XenForo 1.5 and installed Resource Manager version 1.2.5 because the newest version of Resource Manager wouldn't install into XF 1.5, I also installed the prerequisite convert threads to resources 1.0.2.

It appears like the DownloadsII database got imported ok with the importer, 3840 entries in the xf_resource table, looks like All of the categories are there but none of the download links work and the images don't appear. I can't find the files in the internal_data folder, it also appears that the resources were assigned to the wrong Users. I don't think I retained imported content IDs. I also ran the rebuild Resource Caches.

I purchased XenForo and the Resource Manager due to possibly importing the downloads which are important to me, I must be screwing something up, any suggestions?

Thank You

 

[ChicagoCoin]

A update, I started over with a new VBulletin import into XenForo 1.5, installed Resource Manager version 1.2.5 and ThreadToResource_1.0.2.

The new DownloadsII import seemed to go ok, had a couple of server timeouts (guessing) and just refreshed the install page when it was importing the files and it proceeded to go up until about 98% complete and then received this message:
Zend_Db_Statement_Mysqli_Exception: Mysqli statement execute error : Duplicate entry '5460' for key 'PRIMARY' - library/Zend/Db/Statement/Mysqli.php:214

I then looked at the forum and all of the categories and Downloads and Users look like they got imported ok, I was so happy, there were a few entries in the temporary forum that I deleted and then spent hours upgrading settings and stuff. I also stupidly deleted the forum that was used for convert threads to resources, then was going to try and import smilies from VB and was shocked when I went to Import external data and it mentions that a Import is in progress. Tried to continue the import but I'm getting the Duplicate entry errors and I know that the node had been deleted.

It looks like the 3rd step of the Importer was 'DownloadsII Comments', I have some comments imported so perhaps that ran. I discovered that only about 60 files were not imported but everything else looks good and was thinking about just trying to upgrade XF from 1.5 to 2.1.5 and then manually adding the missing files.
If I disable the plugins while the importer is supposedly still in progress, anyone know if I could upgrade XenForo from 1.5 to the latest?

Sorry about the long posts, trying to include a lot of information.

 

[ChicagoCoin]

I think I successfully imported most VB DownloadsII to XF 2.1.7 from 1.5 while the import was still supposedly running at possibly 98% on the importing files process.

My files now have digit codes in front of them when downloading:
Batman 66 VPX becomes 98012-Batman 66 VPX

nobody needs 98012, can I get rid of it?