vBulletin Big Board Importer [vBulletin 3 + vBulletin 4] [Paid] 1.5.0

[Jake Bunce]

I would have to debug it. Shoot me a PM with server login / URL and I can take a look.

Possibilities:

1) Bad $attachFile specified.

2) $isVB4 flag is incorrect (this affects file system id that is used).

3) Missing data file.

 

[Hornstar]

@Jake Bunce

I've tried changing the table names and field names of the thanks import over to dbtech import, and this is what I have done:

// Likes
     $start = microtime(true);
     echo "  post thanks as likes";
     exec('mysql -h' . $this->slaveDbHost . ' ' . $this->sourceDb . ' ' . $this->sourceDBuser . ' ' . $this->sourceDBpassword . ' -N -q -e "SET NAMES binary;SET @like_id = 0;SELECT SQL_NO_CACHE @like_id:=@like_id+1 AS like_id, \'post\' AS content_type, dbtech_thanks_entry.contentid AS content_id, dbtech_thanks_entry.userid AS like_user_id, dbtech_thanks_entry.dateline AS like_date, post.userid AS content_user_id FROM dbtech_thanks_entry LEFT JOIN post ON (post.contentid = dbtech_thanks_entry.contentid) ORDER BY dbtech_thanks_entry.entryid" ' . self::$extraOutCommand . ' > ' . self::$dataDir . 'xf_liked_content.txt');
     echo " (" . number_format(intval(shell_exec('wc -l ' . self::$dataDir . 'xf_liked_content.txt'))) . ' records in ' .  number_format(microtime(true) - $start, 2) . "s)...\r\n";
     }

Just for ease of reference, this is what it was before:

// Likes
$start = microtime(true);
     echo "  post thanks as likes";
     exec('mysql -h' . $this->slaveDbHost . ' ' . $this->sourceDb . ' ' . $this->sourceDBuser . ' ' . $this->sourceDBpassword . ' -N -q -e "SET NAMES binary;SET @like_id = 0;SELECT SQL_NO_CACHE @like_id:=@like_id+1 AS like_id, \'post\' AS content_type, post_thanks.postid AS content_id, post_thanks.userid AS like_user_id, post_thanks.date AS like_date, post.userid AS content_user_id FROM post_thanks LEFT JOIN post ON (post.postid = post_thanks.postid) ORDER BY post_thanks.id" ' . self::$extraOutCommand . ' > ' . self::$dataDir . 'xf_liked_content.txt');
     echo " (" . number_format(intval(shell_exec('wc -l ' . self::$dataDir . 'xf_liked_content.txt'))) . ' records in ' .  number_format(microtime(true) - $start, 2) . "s)...\r\n";
     }
     else if (self::$doVbseoLikes)
     {

Here is the dbtech schema or the table name and field names:


I am such a rookie I am having a hard time getting it right. This is the error I get when importing:
post thanks as likesERROR 1054 (42S22) at line 1: Unknown column 'post.contentid' in 'on clause'

How do I fix this, it's driving me bonkers lol

 

[Hornstar]

Please .

 

[Slavik]

is post.contentid actually there?

 

[Hornstar]

is post.contentid actually there?

No but I'm not sure what it should be changed to. The post. bit confused me.

 

[Slavik]

That would just stay as post.postid I imagine.

 

[Hornstar]

Thanks Slavik, that fixed my dbtech thanks import

I would have to debug it. Shoot me a PM with server login / URL and I can take a look.

Possibilities:

1) Bad $attachFile specified.

2) $isVB4 flag is incorrect (this affects file system id that is used).

3) Missing data file.

Thanks turns out it was a bad @attachFile specified.

@Bossman you will need to make sure you have a forward slash at the end of your file name:

// VB'S STORAGE DIRECTORIES
// IF EMPTY ('') THEN IT LOOKS TO THE DATABASE FOR THE FILE DATA
public static $avatarFile = '/home/site/public_html/forums/customavatars/';
public static $profilePicFile = '/home/site/public_html/forums/customprofilepics/';
public static $attachFile = '/home/site/attachments/';

 

[Bossman]

@Bossman you will need to make sure you have a forward slash at the end of your file name:

// VB'S STORAGE DIRECTORIES
// IF EMPTY ('') THEN IT LOOKS TO THE DATABASE FOR THE FILE DATA
public static $avatarFile = '/home/site/public_html/forums/customavatars/';
public static $profilePicFile = '/home/site/public_html/forums/customprofilepics/';
public static $attachFile = '/home/site/attachments/';

Tried with and without, same result.

 

[Hornstar]

Tried with and without, same result.

Have you checked what the file path to your attachments are set to in your vBulletin admincp? Admincp -> attachment storage type -> it will say:

Storage Type: Attachments
Attachments are currently being stored in the filesystem at /home/yoursite/attachments

If that path matches yours and you have tried both the / and without then double check the vb4 flag is set to false if you are on vb3

 

[Bossman]

Checked, it's all good (and the flag is set to True, because it's vB 4).

What is freaking me out a bit is the fact that I tried the official importer as well, just for kicks. And it behaves exactly the same, but doesn't throw out any errors.

 

[Slavik]

Checked, it's all good (and the flag is set to True, because it's vB 4).

What is freaking me out a bit is the fact that I tried the official importer as well, just for kicks. And it behaves exactly the same, but doesn't throw out any errors.

Try using the inbuilt vbulletin tools to move them back to the database and try from there.

 

[Adam Bloch]

Hi All,

I'm hoping you can help me out - I expect I've got something basic not set as it should be...

When I run the exporter it appears to be working - i.e. gracefully returns me to a prompt, but reports 198 records where there should be millions; when I open up the dump files in the temporary directory, most of them contain 198 lines of the MySQL syntax quick-help.

We'd tried running the exporter on our live box, and found this issue, so I'm now playing in a Debian machine I have at home.

I've dropped extraneous tables from the vB database, so the schema matches a squeaky-clean vB 4.1.10PL3 install (the last build we updated to - long story of incompatible third party code); and of course I can now make any changes to the system config without worry.

I'd be eternally grateful if someone could point me in the right direction...

Thanks in advance,

A

 

[Slavik]

Sounds as if your MySQL install doesn't support all the flags we are using.

You'll have to dig through and find which ones.

 

[Adam Bloch]

Could you recommend a particular MySQL build/package/whatever which does support the flags? (i.e. the version you're actually using)

It's about my luck that both our live server (CentOS & litespeed) and my home machine (Debian 7.3.0 & Apache) have incompatible MySQL builds.

For what it's worth, I just took the latest MySQL & PHP metapackages from dpkg after installing (it's a fresh build done this week).

Thanks for your help,

A

 

[Slavik]

Just installing mysql 5.5 from the base repo on a centos 6.4 clean install has everything you need, or ive never run into any problems.

Percona server 55 works also.

 

[Adam Bloch]

That's magic; I'll get on that.

Sincerest thanks.

A

 

[kontrabass]

I'm wondering if I can get some help with protected function _getReplaceBBCodeCommand() around line 190:

        return ' | sed -r \'s/\[quote="?([^]";]*?);([^]";]*?)"?\]/[quote="\1, post: \2"]/ig;s/\[highlight\]//ig;s/\[\/highlight\]//ig;s/\[nofollow/\[url/ig;s/\[\/nofollow/\[\/url\]/ig;s/\[noparse\]/\[plain\]/ig;s/\[\/noparse\]/\[\/plain\]/ig;s/\[strike\]/\[s\]/ig;s/\[\/strike\]/\[\/s\]/ig;s/\[url\]http\:\/\/www.youtube.*v=([^&[]*?).*\[\/url\]/\[media=youtube\]\1\[\/media\]/ig\'';

I understand that this line is redoing some bbcodes, including youtube. Can someone help me edit this regex so that it includes SoundCloud? I want to rewrite these:

 http://soundcloud.com/luxxury/hotel-california-luxxury-edit

and

[SOUNDCLOUD]http://soundcloud.com/chickennugget/mountain-view[/SOUNDCLOUD]

to:

[media=soundcloud]http://soundcloud.com/chickennugget/mountain-view[/media]

Kier's "Find and Replace" addon is an option, but takes a long time with 14M posts and I'd rather get this done in one pass.

Thanks for any help!

 

[Adam Bloch]

Removed my last post (in the not so wee hours of this morning) to play some more, but...

First of all - I don't want to jump the queue; I wish I could be of some use to kontrabass, but I'm not great at regex at the best of times - never mind on ~3hrs sleep! Sorry!

@Slavik: I'd be grateful if you could share some specific details of the MySQL/PHP config you have which plays nicely with the script... I've spent a good while poking around with various setups, and I keep running into the same problem.

I've tried building CentOS 6.4 with vanilla packages from the base repo, found the same output. I've since tried various other builds/setups (older and newer), and I'm now running a lean & mean CentOS 6.5 install with MySQL 5.6.15 and PHP 5.5.7, and I'm still just getting MySQL quick-help.

Perhaps it's something else I've got wrong?

Sincerely grateful for any advice. My laptop really is looking nervous now.

Thanks,

A

 

[Slavik]

Try using percona 55, if that doesn't work i'll fire up a VM and test myself.

 

[Adam Bloch]

@Slavik - Thanks. I'll give that a go if I don't get any joy with the standard stuff soon.

@ Anyone who has the script reading from the database to intermediate files:

I'd be eternally grateful if you might post your MySQL version/build string, or even just the overall version information for the MySQL Server and Client/Library shown at the homepage in phpMyAdmin. Switching to percona 55 is of course worth a go, but I'd really like to understand why I'm struggling with numerous standard MySQL builds.

Thanks again.

A