1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.1 Rebuilding Cache failed after vb3.8 import

Discussion in 'Installation, Upgrade, and Import Support' started by Mutt, Dec 19, 2011.

  1. Mutt

    Mutt Well-Known Member

    My install was successful & my 23 million post import finally completed. The cache has been rebuilding for most of the day but got stuck here

    Rebuilding Caches...

    Rebuilding... Threads 96,248

    I tried hitting refresh to get it started again but received this message

    An error occurred or the request was stopped.

    I have 2 buttons I can click. the "Continue Rebuilding" button brings me to a blank page w/ nothing happening. The "Stop Rebuilding" brings me to a the rebuild is complete page but I seriously do not think it's over.

    I checked the Server Error Logs & there's nothing from today.

    does anyone have any advice on how to get this bad boy going again? It appears the I can click the "stop rebuilding" button & then head over to "Tools / Rebuild Caches" & get it restarted from there but is there something different I should be trying?

    thanks in advance
  2. Mutt

    Mutt Well-Known Member

    the server seems fine, the drive is not full. I hit back a couple times & the cache starting to rebuild again but got hung up & gave me the error again in about the same spot.

    in case it matters, I checked the DB & it's not what I expected it to be. larger than the VB DB i'm importing but less records. I guess I could have done a double import or something. not sure if this is normal. for example, here are 2 before & after post tables
    vb posts - 22,478,295 records MyISAM latin1_swedish_ci 6.4 GiB
    xen posts - 17,899,246 records InnoDB utf8_general_ci 12.1 GiB
  3. Dan

    Dan Well-Known Member

    Is it stopping at the exact same point every time? I know you said "about", but if it is the exact spot, that could possibly say something.
  4. Mutt

    Mutt Well-Known Member

    That's just it, it wasn't exact. It was 96284 then I backed up & it gave me the error around 91000. I got it going again & that time it reached 96371 before giving me the error. I stopped messing w/ it then worrying that maybe I was making things worse.

    I rain to an error while importing but that was caused by the hard drive being full. I deleted a ton of oversized logs & was able to get it running again. I went and checked again to see if I had bloated logs but there weren't.
  5. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    A blank page is often a suppressed error. Try adding this line to your library/config.php file:

    ini_set('display_errors', true);
    Or add this to a .htaccess file on your server (only works on Apache servers):

    php_value display_errors 1
    That may reveal an error message on the blank page.
    Dan likes this.
  6. Mutt

    Mutt Well-Known Member

    Jake, thanks so much. i'll see what that turns up.
  7. Mutt

    Mutt Well-Known Member

    after making that change, here's the error I'm getting
    Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 39 bytes) in /var/www/virtual/sternfannetwork.com/public_html/xen/library/Zend/Db/Statement/Mysqli.php on line 298
  8. Brogan

    Brogan XenForo Moderator Staff Member

    XenForo sets the memory limit to 128 MB, try increasing it in config.php:
    ini_set('memory_limit', 256 * 1024 * 1024);
    Or set it to -1 to remove the limit altogether.
  9. Mike

    Mike XenForo Developer Staff Member

    He's unlikely to be able to change the limit based on that error, as it should already be at least 128M. The change will have to happen in php.ini.
  10. Mutt

    Mutt Well-Known Member

    I made this change & it's going again.!! if it craps out again I'll try making the change in php.ini
  11. Mutt

    Mutt Well-Known Member

    i made the change to php.ini & rebooted. it's working now. 256M wasn't enough but 512M seems to have done the trick. at least for now.
    Dan likes this.
  12. Mutt

    Mutt Well-Known Member

    it says it finished. I'm still concerned about what I'm seeing in phpmyadmin. thread records are only slightly decreased so maybe we lost 3,500 threads out of 588,000. not too worried about that. but what the hell happened to the posts? it looks like 1/2 the posts (11 million) are gone but for some reason the 1/2 half that made it are taking up twice the space as the original instead of 1/2. certainly seems broken. even if the import was merging posts to account for the smaller size, why the larger file. and if the larger file was cause by threads getting imported twice, why the smaller record count? i'm totally confused on this one.

    thread - 588,700 records - MyISAM - latin1_swedish_ci - 179.2 MiB
    post - 22,478,295 records - MyISAM - latin1_swedish_ci - 6.4 GiB

    xf_thread - 585,111 records - InnoDB - utf8_general_ci - 137.2 MiB
    xf_post - 11,855,347 records - InnoDB - utf8_general_ci - 12.1 GiB
  13. Mutt

    Mutt Well-Known Member

    next up, should I do the search index now or am I wasting my time until I get the posts straightened out? or is the upcoming big board search around the corner & won't require it?
  14. Mike

    Mike XenForo Developer Staff Member

    You can't compare sizes between InnoDB and MyISAM. InnoDB takes more space in general.

    As for the counts, as I noted, you can't trust an InnoDB count in phpMyAdmin/MySQL metadata unless you actually run a (very slow) query to do a proper count. Alternatively, you had orphan data that wasn't imported. I'd look around for missing data.

    MySQL full text search really won't be usable with that much data. I probably wouldn't even bother indexing the old content unless it's going somewhere else.
  15. Mutt

    Mutt Well-Known Member

    thanks mike.. I wasn't sure how I'd find missing post among millions that made it but if this isn't that unusual, I'll just move on.

    I was thinking i'd install apache locally & redo the import here to see if I got the same results. Should I even bother?

    and thanks about the search index. Is there anyway to limit the searxh index to just the last years worth of posts or something like that where the newest stuff would still be searchable?
  16. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

  17. Mutt

    Mutt Well-Known Member

Share This Page