XF 1.5 Migrate/Copy from XenForo to.. XenForo

Luchian

Member
Hi guys,

I am in the admin team of a small community. We used to have a dedicated sub-forum on a bigger XF forum (A).
However, the community has grown, we have purchased a new domain and have installed XF 1.5 (new license) on the new site (B).

I would like to know if it is possible to copy/migrate our original sub-forum from old forum (A) to the new forum (B)?
And if yes (hopefully), what would be the safest way on doing it? (It would mean a lot to be able to do it).

To spice it up:
The new forum (B) started to get populated. So db are not empty anymore.
Some members, now have an account both on the old (A) and new (B) forum.

Thank you in advance.
 

Brogan

XenForo moderator
Staff member
It's not possible to import a subset of data.

You would have to do a complete import and then delete the content you don't want.

The importer will provide the opportunity to merge duplicate accounts with the same user name or email address.
 

Luchian

Member
Thank you Brogan.

I have looked a bit in the admin panel. I see there is an import option in the Tools tab.

But noob question:
Do I need to export the old forum in some friendly format?
Or do I directly copy the databases and use those?
 

Luchian

Member
Thank you for the info.

We plan on taking the dive somewhere next week.

If the old forum has some 14k users, and we only wish to transfer about 300 of them, does that mean that we need to import all anyway, and then delete those we do not want? :oops:
 

Sim

Well-known member
Thank you for the info.

We plan on taking the dive somewhere next week.

If the old forum has some 14k users, and we only wish to transfer about 300 of them, does that mean that we need to import all anyway, and then delete those we do not want? :oops:

Alternative approach: duplicate your old forum, delete all content and users you don't want to migrate across, then do the import.

I'm about to go through pretty much the same exercise this week - merging an older XF1.5 forum into a newer XF1.5 forum.

My plan is as follows:
  • create a full copy of the old forum on my dev machine (running in VirtualBox) in case I ever want to access old content
  • clean up the old forum to remove any content and users I don't want to import
    • hard delete old threads which are not relevant to new site
    • delete forums/nodes which are not relevant to new site
    • delete any thread prefixes that are no longer in use
    • remove custom user profile fields which won't be used on new site (I am going to look at creating a new custom field which is set for every user on the old forum to help me identify users who came across from the old site - haven't tested that yet)
    • remove custom user groups which are no longer relevant
    • demote moderators/admins
    • rebuild user caches to ensure previous changes to post counts are reflected in database
    • delete all users who are: awaiting email confirmation | awaiting moderation
    • delete all users who are: awaiting email confirmation (from edit) | email invalid | banned - where post count = 0
    • delete all users who are valid with post count = 0 and last visited more than 6 months ago
  • create a test copy of my new forum on my dev machine
  • do a test import of the cleaned old forum to check things are going to work and identify anything I may have missed in my cleanup
  • do the import on the production version of my new site (after full backups, etc!)
  • turn off the old forum
  • tidy up the old content on the new site
  • once I finish migrating all of my custom addons, upgrade the new forum to XF 2.1 :D
 

STNN

Active member
Thank you for the info.

We plan on taking the dive somewhere next week.

If the old forum has some 14k users, and we only wish to transfer about 300 of them, does that mean that we need to import all anyway, and then delete those we do not want? :oops:

Wait, you don't have to do that.

I just migrated from phpbb to xenforo like few weeks ago, something with posts went horribly wrong while removing the broken bbcode of phpbb and no way to fix it. There was lots of new content and members so i couldn't use the xf importer again. I just wrote a simple script copying posts from phpbb to xenforo and done! It was hundred thousand posts!

Xenforo to xenforo should be a piece of cake. You will probably have different thread/userids from forum A to B but it still beats having to lose members and content outright.

I can share my script i used for copying from phpbb to xf, maybe you can understand it but really all it was, was select and update mysql queries and convertphpbbcode.

Let me know if you need help, i can't promise doing it all myself since i have way too much to do myself but can certainly guide you.
 

Luchian

Member
Alternative approach: duplicate your old forum, delete all content and users you don't want to migrate across, then do the import.

I'm about to go through pretty much the same exercise this week - merging an older XF1.5 forum into a newer XF1.5 forum.

My plan is as follows:
  • create a full copy of the old forum on my dev machine (running in VirtualBox) in case I ever want to access old content
  • clean up the old forum to remove any content and users I don't want to import
    • hard delete old threads which are not relevant to new site
    • delete forums/nodes which are not relevant to new site
    • delete any thread prefixes that are no longer in use
    • remove custom user profile fields which won't be used on new site (I am going to look at creating a new custom field which is set for every user on the old forum to help me identify users who came across from the old site - haven't tested that yet)
    • remove custom user groups which are no longer relevant
    • demote moderators/admins
    • rebuild user caches to ensure previous changes to post counts are reflected in database
    • delete all users who are: awaiting email confirmation | awaiting moderation
    • delete all users who are: awaiting email confirmation (from edit) | email invalid | banned - where post count = 0
    • delete all users who are valid with post count = 0 and last visited more than 6 months ago
  • create a test copy of my new forum on my dev machine
  • do a test import of the cleaned old forum to check things are going to work and identify anything I may have missed in my cleanup
  • do the import on the production version of my new site (after full backups, etc!)
  • turn off the old forum
  • tidy up the old content on the new site
  • once I finish migrating all of my custom addons, upgrade the new forum to XF 2.1 :D
Thank you for the checklist, the offline cleaning can be also an idea indeed.

Wait, you don't have to do that.

I just migrated from phpbb to xenforo like few weeks ago, something with posts went horribly wrong while removing the broken bbcode of phpbb and no way to fix it. There was lots of new content and members so i couldn't use the xf importer again. I just wrote a simple script copying posts from phpbb to xenforo and done! It was hundred thousand posts!

Xenforo to xenforo should be a piece of cake. You will probably have different thread/userids from forum A to B but it still beats having to lose members and content outright.

I can share my script i used for copying from phpbb to xf, maybe you can understand it but really all it was, was select and update mysql queries and convertphpbbcode.

Let me know if you need help, i can't promise doing it all myself since i have way too much to do myself but can certainly guide you.
My co-admin has some coding-fu, could be useful to take a look at your script. Thank you for your proposal.
 

STNN

Active member
My co-admin has some coding-fu, could be useful to take a look at your script. Thank you for your proposal.

PHP:
<?PHP
set_time_limit(0);

ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);

@$db_server = "localhost";

// Connect to Database function
function connect($db_user, $db_pass, $db_to_select)
{
global $db_server;
$link = mysqli_connect("127.0.0.1", $db_user, $db_pass, $db_to_select);

if (!$link) {
    echo "Error: Unable to connect to MySQL." . PHP_EOL;
    echo "Debugging errno: " . mysqli_connect_errno() . PHP_EOL;
    echo "Debugging error: " . mysqli_connect_error() . PHP_EOL;
    exit;
}

return $link;
}

$mconnect = connect("","",""); //stripped my db info


$query = "SELECT post_id, post_subject, post_text FROM pbcef_posts ORDER BY `post_id` ASC LIMIT 130000,20000"; //ORDER BY `post_id` ASC LIMIT 0,30";

$result = mysqli_query($mconnect, $query);
if(!$result)
 {
         echo "Error: Unable to connect to MySQL." . PHP_EOL;
 }
 $row_cnt = mysqli_num_rows($result);
    printf("Result set has %d rows.<br /><br />", $row_cnt);

$i = 1;
    /* fetch associative array */
    while ($row = mysqli_fetch_assoc($result))
    {
        
$subject = $row["post_subject"];
$subject = mysqli_real_escape_string($mconnect, $subject);

$posttext = $row["post_text"];
$postid = $row["post_id"];
$posttext = html_entity_decode($posttext);

$blah = new PhpBb();
$posttext = $blah->convertContentToBbCode($posttext);
$posttext = mysqli_real_escape_string($mconnect, $posttext);

/*echo $subject . '\r\n';

echo "<br /> <br />";*/

//echo $topic_title . "$postid";
    if ($postid >= 135671){ // After this ID, new content posted after import starts. Don't want to overwrite that.
    return;
    }
echo $postid;
echo "<br />";
//$post_text = nl2br($post_text);
//str_replace("'", "\'", $post_text);
//$post_text = addcslashes($post_text, "'");

//$post_text = mysqli_real_escape_string($mconnect, $post_text);

$sql1 = "UPDATE `xf_post` SET `message` = '$posttext' WHERE `post_id` = $postid";
//echo "<br />";
//echo $sql1;
//echo "<br /><br />";
$result1 = mysqli_query($mconnect, $sql1);
if(!$result1)
 {
         echo "Error:." . PHP_EOL;
 }
//usleep(200);

    }

    /* free result set */
       mysqli_free_result($result);

/* close connection */
mysqli_close($mconnect);

?>

It's all just mysql select and update query. I stripped the Xenforo importer bbcode code because i don't think i can post it in public.

If forum A and B both have same version of Xenforo then just simply copy rows from forum A, move the id(userid, topicid) around to an unoccupied id in destination (B) forum table and done. The moving of ID is going to be the tricky part, rest is simple copying.
 

Luchian

Member
Thank you @STNN . Will ask my friend to have a look and see if he can figure it out.

***
A new question, for the "classical" method of importing directly the db:
Old forum (A) has the Resource Manager installed, with uploaded resources assigned to their respective threads.
The new forum (B) has no RM - and there is no intention to install one at this point.

Can this create an unexpected problem/behavior when trying to import the DBs? :unsure:
 

Luchian

Member
Hi guys, me again.

Since my friend does not have to much time on his hands for experiments, I need to figure this on my own 🤓. Therefore, with the risk of being bullied for the noobness, could somebody please detail what exactly do I need to fill in the paths for data directories? 😇

1/where do I copy the db?
2/the path to Data Directory relative to what?
3/same question for Internal Data Directory

Notes:
I've successfully installed XAMPP and XF 1.5.x on my PC.
Then plan is to first import the db locally, clean it, and then do the final import on the B forum.
I have the db, the user and pwd.

Many thanks to the kind knowledgeable guy person assisting.
 

Attachments

  • 2020-07-06_190544.png
    2020-07-06_190544.png
    18.1 KB · Views: 5

Brogan

XenForo moderator
Staff member
If you are using xampp locally, download the database dump to your computer and import it using phpmyadmin built in to xampp.
Or use SSH/the terminal if it is a big database.

The directory paths of the forum you are importing are relative to the installation you are importing into.
If you aren't sure what they are, the easiest way is to just copy the data and internal_data directories to temp directories inside the installation you are importing into.

For example:
root
data​
internal_data​
temp_data​
temp_internal_data​
 

Luchian

Member
Decided to go with your first proposed solution. Seems to be working up to a point :).

I get this error:
2020-07-09_235145.png

After a little googling, I see that there might be 2 possible solutions (not sure which to choose?):

1/ just delete the line from my sql db?

2/ add some lines (e.g. below) at the beginning of my sql db?
Code:
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=[spam-filter]CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=[spam-filter]CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=[spam-filter]COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=[spam-filter]TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=[spam-filter]UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=[spam-filter]FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=[spam-filter]SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=[spam-filter]SQL_NOTES, SQL_NOTES=0 */;


=================================
For info, this is how the db contents currently look:
2020-07-10_000830.png
 
Top