• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

vB3.8 Migration - QUOTES

yoghurtfarmer

Well-known member
#3
This raises the question.. is there a tool included with xenforo to clean up posts after import? ie: a xenforo analogue to impex's cleaner.php
 

dutchbb

Well-known member
#4

brendanc

Active member
#7
I'm going to run into this issue myself when I import, so if there isn't an official fix by that time, I'll write a simple php script that will scrub the database and replace where necessary.
 

kyrgyz

Well-known member
#8
I'm going to run into this issue myself when I import, so if there isn't an official fix by that time, I'll write a simple php script that will scrub the database and replace where necessary.
Since Mike and Kier keep numb about quote conversion, I am assuming it'll be an issue. Could you please share your script with us if the train goes south?
 

brendanc

Active member
#9
Since Mike and Kier keep numb about quote conversion, I am assuming it'll be an issue. Could you please share your script with us if the train goes south?
I absolutely will. I've been doing data migration stuff at my day job as well, so it's still pretty fresh.

There's no way my forum could get along without quotes, we use them *constantly*.
 

Mike

XenForo developer
Staff member
#10
It's not possible to change the IDs when importing, as they're simply not guaranteed to be known. (I suppose we could try, as the average case would be known.) It's not a matter of just modifying the quote format, but looking up every post ID and replacing it. This would have to be done via a second pass script, that would take a significant amount of time on a large board.
 

Paul M

Well-known member
#11
If the posts were imported in date order, wouldnt you always know, since by definition, it must have quoted an older post ?
 

brendanc

Active member
#12
It's not possible to change the IDs when importing, as they're simply not guaranteed to be known. (I suppose we could try, as the average case would be known.) It's not a matter of just modifying the quote format, but looking up every post ID and replacing it. This would have to be done via a second pass script, that would take a significant amount of time on a large board.
I think that's absolutely worth it. I could give two craps as to how long an import takes as long as my data is imported and referenced properly.

I say: do it right. Even if it means some people will have a "lengthy" import process. That's the genius about an import, though; you only have to do it once.

It really wouldn't be that difficult I don't think. If you guys don't want to spend the time doing it, please make a table for users who import which maps old forum IDs to XF ids.

Code:
CREATE TABLE `xf_posts_map` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `old_post_id` int(11) DEFAULT NULL,
  `xf_post_id` int(11) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
From there I am more than happy to do the post scrubbing script.

Honestly if there's anything you need to do right, it's the import. vB's Impex has this down to a science, and there's no reason you guys can't pull it off as well, especially considering the schema is extremely similar (according to Kier).
 

Kier

XenForo Developer
Staff member
#13
and there's no reason you guys can't pull it off as well, especially considering the schema is extremely similar (according to Kier).
Eh? I think you have misunderstood me.

I have said that the vBulletin 4 schema is very similar (well, virtually identical) to the vBulletin 3.8 schema, but neither are anything remotely close to the XenForo schema.
 

brendanc

Active member
#14
Ah. Maybe I did misunderstand you.

Anyhow, giving it some thought though, I don't see why you couldn't create the mapping table for which to migrate the data. During the post import (and the thread import as well, I would like to be able to redirect my old vB urls to the new XF ones, for SEO purposes), after each insert query, use
PHP:
mysql_insert_id()
to get the XF post/thread ID and just insert a row into the proper mapping table.

Normally I wouldn't suggest using mysql_insert_id(), but since this is during the import there is no risk of having other inserts running. I'm more than happy to test out/write some code if you guys wanna toss over the import script/db schema. I want this to be ready asap just as much as you guys do.

With the mapping table in place, we can create a cleanup script that will scrub the data and replace the incorrect IDs with the correct ones. (and maybe add an option that deletes the mapping tables when you're done importing)
 
F

Floris

Guest
#18
Hi Floris,

Wouldn't it be better if the were?
Yeah, would be nice, but they won't. Because you're perhaps making something, making usergroups, making forums, and some threads, users .. and importing on top of it. So anything already with a uniqueid for a user, group, forum, thread or post ... will error with "duplicate key".

You're importing on Top of something. So you're importing new id, which goes auto increment ..

Your old board might end with unique id 15000 for posts, after deleting 10000 posts first. Your new board latest id might be 5010.
 

AndyB

Well-known member
#19
Thank you for the explanation Floris. I can see that it would be much easier to auto-increment id's.

How does the lookup work so that old links to your forum are redirected properly? I assume there is a table that is searched and if a match is found it will redirect to the new post, thread or attachment.