1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Duplicate Threads?

Discussion in 'Installation, Upgrade, and Import Support' started by Mopquill, Oct 9, 2010.

  1. Mopquill

    Mopquill Active Member

    Okay, so, I'd installed XenForo, and imported my vBulletin 3.8 settings just fine. However, due to a very long story, one of my staff accidentally wiped the database. I hadn't made a backup yet (D'oh!), but it'd only been a few days, so, I figured I'd just re-import vBulletin, as it's been "off" and unused since the first import (i.e. the exact same import I did the first time).

    Now though, when I import, everything proceeds fine as far as the import itself goes. If I check the forums root then, it says there are 86,000-something messages. Once I proceed with the import completion (Click the "Complete Import" button), it goes and rebuilds caches. Then, it tells me to do two things: 1. check the usergroup permissions, and 2. rebuild the search cache. But, at this point, the forums say there are 168,000-something messages. If I view any forum, I can see two of each thread all the way down.

    I've tried dropping every single table in my database and starting the process over- three times. This keeps happening. Anyone have any ideas? I've completely re-uploaded the XenForo files already, and wiped the database, and even used a different (new) database/user.

    The Error log in the AdminCP is empty. I'm thinking there might be some cache file somewhere ruining something, so, I figure I might need to delete the /forums/ directory, wipe the database, re-upload it, and re-install. Anyhow, this seems to be indicative of some bug or glitch or something. I don't want to have to go through my board and delete some 3,000 threads (and have all the orphaned thread/post IDs to go with that), so, I'm wondering if anyone has come across this, and has any potential solutions. I'm willing to try anything that sounds intelligent.

    At the very least, thanks for reading.

    EDIT: Not sure if this is relevant, but, I'm adding it in:
    The staff member was trying to install their own xenforo forum under a different linux user on a different website, and somehow got MY forum's database information in their config.php . I've had them check and double-check, but, they have no idea how it happened. I'm sort of willing to blame bumbling, but, it sort of seems like something else is happening here.

    Considering that wiping the database and files should leave everything exactly the same as the first install and it's -not-, that would indicate to me that something it can read somewhere changed. I'm not sure what it changed, though.
  2. Mopquill

    Mopquill Active Member

    Anyone? This is really important, and I have little idea about what to do. :X
  3. Mopquill

    Mopquill Active Member

    I've completely wiped all files, uploaded them from scratch, started a new database entirely, etc. etc. This is still happening, all the threads are still being duplicated. I'm hoping the records will be in order from the duplication, so I can delete N - N and be fine, and hopefully preserve my ID hierarchy.

    Anyone, PLEASE let me know if you have any idea why this is happening. Does XenForo write to the database you're importing from in any way that would affect this?
  4. thumped

    thumped Well-Known Member

    i've seen this too. import - delete xf db - import again = duplicate threads

    sorry, i know that's not exactly helpful
  5. Mopquill

    Mopquill Active Member

    This is horrible. I found a sort-of solution:

    SELECT DISTINCT user_id, message, post_date FROM xf_post LIMIT 999999;

    make sure 999999 is greater than the number of posts you have (counting excess!). However, XF just adds them back again!

    I think we need a dev in here. =[

    EDIT: Updated my first post.
  6. Mopquill

    Mopquill Active Member

    Actually, I think I found a solution.

    First, make sure your database is utf8_general_ci , or the new table you create will not be.

    To do that, it's:
    replacing dbname with your database name.

    Run the query:

    CREATE TABLE xf_post2 as
    SELECT * FROM xf_post WHERE 1 GROUP BY user_id, message, post_date;
    then, run the query:
    DROP TABLE xf_post;
    then, run the query:
    RENAME TABLE xf_post2 TO xf_post;
  7. Mopquill

    Mopquill Active Member

    It seems that while my query works, I can't get it to discriminate and delete the higher post_ids over the lower ones. In the grand scheme, that might not matter, but, I'm picky. Although, it may matter if corresponding threads and posts aren't the same (i.e. post id 1 and 87000 are the same post, and 1 gets preserved over 87000, but thread id 1 gets wiped, and thread id 3600 gets saved).

    Besides that, I can't seem to get the caches to rebuild. :-/
  8. Mopquill

    Mopquill Active Member

    I've found a few *solutions* to this, but, none of them leave me with an adequate amount of faith in my database integrity. I'm worried about orphaned posts, specifically. I'm not a MySQL pro, it'd be nice if someone could shed some light on this, or if someone could tell me they have identified a bug in the import script and are working on a fix, or something.

    I even restored a backup of the original vB database from BEFORE XenForo accessed it, meaning if there was anything that got put in there, it wouldn't be in there. Same problem. I'd really like to know what's up. :-/
  9. cedivad

    cedivad Active Member

    Getting the same problem.
    Do you say that changing the collocation from latin1 to utf8 will fix this?

    Mike or Kier is welcome here, at least to discover what's wrong so that we can convert our forums =)
  10. Mopquill

    Mopquill Active Member

    No, no. Changing the collation only lets you run that query and not get a botch table in a different collation than it should be. For instance, if I make a database and don't specify, it comes out latin1_swedish_ci. So, if I remove a utf8 table in that database and recreate it, it's now a latin1_swedish_ci table with utf8 information.

    Anyhow, everything I've been able to do seems to just be a workaround, and I lose id integrity at the least, as well as potentially having things be broken, though, I likely wouldn't be able to tell until it would be a crucial point.

    Basically, it just needs to import correctly in the first place.

    Also, I was looking through the database, and it seems to be importing the duplicates shortly after their originals. So thread ID 150 might be around 158 or 200 or so as a duplicate. It's nothing as simple as deleting half the posts I have WHERE thread_id > X or anything.

    I tried looking at the php script, but, I can't find where it declares the function "import" (I suspect t might be part of Zend or something), and quite frankly, a lot of it is over my head.

    Thanks for posting, though! I'm really happy to hear other people having the problem. That can hopefully get some light shed on this. My users are going nuts at me over facebook/MSN/AIM, etc.
  11. cedivad

    cedivad Active Member

    I've just finished editing ./library/XenForo/importer/vBulletin.php
    That's the file that need to be edited, i added some test querys. If it works i will share it ;)
  12. Mopquill

    Mopquill Active Member

    Alright, thanks. I was looking in that file, too. Unfortunately, I wasn't able to tell if anything was wrong. :-/
  13. cedivad

    cedivad Active Member

    I got it. The problems seems to be on the javascript. And the fix is... disable it!
    Star the import, before pressing the Threads & Post button, disable JS on your browser. You will have to go on manually, but it's worth the wait!
  14. Mopquill

    Mopquill Active Member

    You're kidding. XD

    Thanks, I will try this. That might be why it's a re-import. A faulty cookie or something. Good catch, if you're right. =]
  15. Mopquill

    Mopquill Active Member

    Ugh. That didn't do it. And it took forever.

    I hope the import script gets fixed soon. >.<
  16. cedivad

    cedivad Active Member

    It looked to work with me...

    The question is... do the staff knows about this?
  17. Mopquill

    Mopquill Active Member

    I hope it did! I'm just saying that it failed for me. I had JavaScript off for the entire import, as well, since I had doubled messages and such in addition to posts/threads.

    I'm sure they know, which is likely why no one is answering. Or they are just very busy. I'm to understand Beta 2 comes out soon.
  18. cedivad

    cedivad Active Member

    I really hope beta 2 to fix this.
    However i just retried doing all the stuff, and you are right: js don't fix it. It would have been too great... I was wrong.
  19. cedivad

    cedivad Active Member

    Ok, i've fixed.
    Or at least i think so. Overwrite the vBulletin.php file with the one you find attached.

    Schermata 2010-10-10 a 21.16.34.png

    Attached Files:

  20. Dean

    Dean Well-Known Member

    Just to note.

    I did pass a link to this thread on, and in addition a few of us are trying to help as best we can. I will try to replicate using my own database at some point.

    The only suggestion I can make at this time is to keep this thread as succinct and as easy to read as possible, which should allow others to help more easily.

Share This Page