XF 1.2 importing from vb with hex chars

[briansol]

I can't seem to get my post data in... i keep getting sql errors from hex codes.

Mysqli statement execute error : Incorrect string value: '\x81@\x81@ne...' for column 'message' at row 1
Mysqli statement execute error : Incorrect string value: '\x81@\x81@it...' for column 'message' at row 1

i've scrubbed my source db as well as i could, replaced all 0X's with Ox's, a search for 81@ne returns nothing but a few email addresses. I can't figure out where its coming from.

I am running the web import script from vb3.8.5.

anyone have any tips?

 

[Mike]

In general, this probably indicates that the data is being converted to a different character set unexpectedly. If you're vB forum is still up and running, can you send a link to it? (If it's not running or you'd rather not, do you know what character set you're using in your default language?)

When the import is started, there's an option to force the character set. You may need to force that to "latin1" or "utf8", depending on server configuration. (Unfortunately, this will need the import to be restarted; to wipe out the existing data, you'll need to do a fresh install or restore to a pre-import backup.)

 

[briansol]

Thanks for that. I've restarted about 8 times at this point :X

The forum is still up, but its been moved and is off, so i'm not sure what you'd need to see. The post (vb) table is in english, post table is latin1_sweedish_ci and the xf_post table is in utf8_general_ci

What should i force it to? destination charset, ya?

 

[briansol]

trying it with the source (as that is where the option lies) of latin1 first. loading users... back in an hour :X

 

[briansol]

well, latin1 fails too :/
Mysqli statement execute error : Incorrect string value: '\x96\x0A Tr...' for column 'message' at row 1

 

[Jake Bunce]

Also check the collations in your XF database. Example:

http://xenforo.com/community/thread...red-please-try-again-later.14832/#post-194621

Your error message appears to name the xf_post.message field.

 

[briansol]

utf8_general_ci
is set on post.message field

i'm running with utf8 right now as the force, and so far so good... i haven't seen .72% yet, so it might just roll through this time.

 

[briansol]

3%. pretty sure this will go to the end now (at least for this error case)
so, anyone else reading, charset force should match xf's destination table collation

 

[Mike]

so, anyone else reading, charset force should match xf's destination table collation

Just to clarify, not necessarily -- that's why I was asking you about the character set chosen in vB's language options (which is different than the collation of the table/column). In most cases where this comes up, forcing "latin1" helps, but there have been a few "utf8" times. Unfortunately, it's not really something we can workaround in an automated way, as there a lot of server settings involved (and it's doubly tricky when a server move is involved, which may have different settings than the previous server).

However, it does sound like "utf8" is working for you. I suspect your forums were already in UTF-8? (So MySQL was doing a conversion back to Latin1 by default, which confused the importer.)

 

[briansol]

My forum was in latin.

What I think threw it off was posts that had windows errors embedded in them with the blue screen hex X-codes. It may not come up on sites that don't have posts like this.

up to 19% now so i'm pretty sure we're good

 

[briansol]

To completion! Thanks for the help guys.