Adopt the Digital Point importer as standard

[Stuart Wright]

http://xenforo.com/community/resources/a-r-f-i-a-really-fast-vb4-importer.1397/
Being worked on by Slavik.
Please integrate this importer into the standard product.
In order to set up AVForums on Xenforo, I would like to import a copy of the data so we can fully test it and the new functionality we will be writing on a large database.
And when it comes to upgrading, closing the forums for an hour while we migrate is of course way less painful.

 

[emc2]

ditto..

 

[klyde]

+1

 

[Paul]

+1

 

[CTXMedia]

Definitely a plus point for getting larger (vB) forums over to XF ...

 

[MGSteve]

+1 we also need the importer to migrate Paid Subs over as well, as much as possible.

 

[Forsaken]

Unless you like the first post your +1 statements don't mean anything.

 

[Rigel Kentaurus]

It might be close to impossible to have that importer as an standard out of the box feature. The importer, as described, runs on CLI (a hurdle for most people), then it requires customization, and it is not the standard database read & write but builds files, parses, then batch imports, and I can see that going really bad with different server configurations (might work in one Linux Distro, definitely not on windows, maybe on Mac ... )

So, as far as making it work for "everybody", I think I would re-read the text in red at the Digitalpoint site

 

[Jarod]

+1

 

[Digital Doctor]

What about Waindigo's ?
http://xenforo.com/community/resources/import-tools-by-waindigo.1255/


Ideally Xenforo would adopt some of the features of his importer ...

Description:

This add-on adds a number of new options when performing an import from another forum software.

By allowing you to undo steps, skip steps and recover previous imports, and by skipping over any IDs that have previously been imported when you run a step more than once, this add-on makes debugging an importer or performing a staged import easier than ever.

This add-on is an advanced tool. Always backup before performing an import. It is also recommended to run a full test import first before completing an import for real.

I think lots of people would like to be able to redo steps, etc.

 

[Paul]

So, as far as making it work for "everybody", I think I would re-read the text in red at the Digitalpoint site


View attachment 42310

I believe Slavik was implementing some stuff so that it would function like a standard migration tool to work with the "standard product"... as per first post by Stuart....

 

[Stuart Wright]

As I understand it, Slavik has been working on creating a more generic version of the importer.
But yes, where certain specific server/setup requirements are needed, then perhaps the importer can check for those and refuse to run unless they are satisfied?

The background for us is that some year(s) ago I did a test import which failed after 48 hours. Obviously from 48 hours down to less than one is a significant improvement

 

[rdn]

What about Waindigo's ?
http://xenforo.com/community/resources/import-tools-by-waindigo.1255/


Ideally Xenforo would adopt some of the features of his importer ...



I think lots of people would like to be able to redo steps, etc.

I support this.

 

[RobParker]

I don't think it makes any sense to make this "standard" as anyone who needs it is a pretty extreme use case and also requires a fair amount of technical expertise compared to a "standard" user. I think making it more generic (as Slavik has done) is extremely useful for people.

The danger of making it a part of the core is that a lot of new users who neither need it or properly understand how to use it would then have problems which would need support.

 

[Stuart Wright]

Ok by 'Standard Product', I mean can it be adopted by the XF team, released and supported officially, as an additional resource if not in the standard product download?
I know naff all about server configuration and don't touch MySQL other than through phpMyAdmin and I leave all that stuff to Tim at Nimbus Hosting as I'm too busy running the business and playing Black Ops II.

If there is a way to package this in a way that dummies like me can have a go with it, then that would be superb.

We're likely going to need additional importing steps to import some additional data like iTrader ratings (if it's not imported by the addon), but an official importer will be good for us, many other forums and indeed Xenforo. How many forum platforms can boast importing ten million plus records in an hour or less? Awesome.

 

[shawn]

There is also a XF-written CLI, but it's also essentially unsupported/beta. Maybe there's a market for XF to offer customized import services instead. I don't know that it's realistic to think that the stock importer should be able to work quickly on +10MM post forums, especially if that importer is pushing the data through the stock XF datawriters.

Waindigo's add-on is for web-based imports. I'm not sure what the realistic limitations of the web importer is, but I can't imagine it's much beyond 2MM posts. I did just shy of 1MM posts, but it took something like 8-10 hours.

 

[klyde]

Ok by 'Standard Product', I mean can it be adopted by the XF team, released and supported officially, as an additional resource if not in the standard product download?
I know naff all about server configuration and don't touch MySQL other than through phpMyAdmin and I leave all that stuff to Tim at Nimbus Hosting as I'm too busy running the business and playing Black Ops II.

If there is a way to package this in a way that dummies like me can have a go with it, then that would be superb.

We're likely going to need additional importing steps to import some additional data like iTrader ratings (if it's not imported by the addon), but an official importer will be good for us, many other forums and indeed Xenforo. How many forum platforms can boast importing ten million plus records in an hour or less? Awesome.

I also have to import a forum from vb4 with more than 16 millions of posts and i would like to do it as fast as possible. I will wait until 1.2 version to see the improvements. Migrations between forum software is not a easy decision.....

 

[Stuart Wright]

Migrations between forum software is not a easy decision.....

From vB3 to XF is an easy decision to make, it's the doing of it which is not so easy

 

[Slavik]

I believe Slavik was implementing some stuff so that it would function like a standard migration tool to work with the "standard product"... as per first post by Stuart....

All going well it should be fully working by the end of the week, just need some more datasets to test it on. though it requires a pretty intimate knowledge of the importer itself, as various things (and core xenforo files) need modifications.

I will be packaging it up with some information for the end user to use, and will probably be looking to sell it (unsupported) for around $150 I imagine as the time investment in it was pretty massive to be honest (much longer than I realised until I totaled it all up) or as a service on a quote-basis.

 

[Stuart Wright]

I'm happy to volunteer the AVForums database as a test subject provided, of course, we can ensure user data privacy.