1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Import with the same ids as former product

Discussion in 'XenForo Development Discussions' started by Marcus, Sep 18, 2011.

  1. Marcus

    Marcus Well-Known Member

    If you import from another forum software to an empty xenforo, it can be useful to have your former IDs. Notice: This thread is for discussing this. There are two ways: The elegant one where everything is done in the importer. I'll go the non-elegant route: Create fake data in the old installation that will be imported from xenforo.

    1 Identify the IDs that are important for you. For my vb board these are: userid, postid, threadid, forumid.

    2 Create rangeid table

    a) create a table "rangeid" with the fields "id" (auto-increment, primary) and "anyid".

    b) auto-fill this table with IDs: "INSERT INTO rangeid (anyid) SELECT postid AS anyid FROM post" and let this query run multiple times

    3) Input fake data into the tables (forum, thread,user,post)

    a) Remove autoincrement and then the primary index from the table you put fake data in

    b) Insert the missing IDs with fake data. Example for userids on vbulletin installation:
    INSERT INTO user (userid,username, email)
    SELECT id AS userid, CONCAT( rand( ) , "removeuser" ) AS username, CONCAT( rand( ) , "removeuser@mail.com" ) AS email
    FROM rangeid
    LEFT JOIN user ON user.userid = rangeid.id
    This query takes a long time so any advice to rewrite it will be very welcome.

    -> now repeat step 3 and d for all other IDs, change the fake data accordingly.

    4 Import as usual from xf admin center.

    5 Remove fake ids Example for userid
    DELETE FROM user WHERE username = "removed later"
    - repeat this step for all other IDs
    Fuhrmann likes this.
  2. digitalpoint

    digitalpoint Well-Known Member

    I've started working on one that retains important IDs as well as being ultra fast (I did a test import of 2.4M threads and 16.8M posts, and it took 28 minutes).

    It's *not* for the faint of heart though. I'm building it really for myself in mind, so it makes certain assumptions... vB site uses UTF-8 as Content-Type, but the text columns are latin1 (vB default). With making certain assumptions about the source setup, we can simply bypass the data writers and just do direct INSERT SELECT straight from source DB into XF.

    Did I mention it only took 28 minutes for 16.8M posts and 2.4M threads? Oh yeah, I did. :)

    It's not anywhere remotely close to being ready for production though... I literally started working on it last night.
    Jeremy, Kier and Marcus like this.

Share This Page