1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Partial Fix [performance] Performance slow when adding a nodes with large number of nodes in system

Discussion in 'Resolved Bug Reports' started by Mike Tougeron, Feb 13, 2012.

  1. Mike Tougeron

    Mike Tougeron Well-Known Member

    When there are a large number of nodes in the system, in this case > 500, adding a new node to an active system can be very slow and frequently timeout before the node is fully created.

    From what I can tell, it will create the record in xf_node but it won't properly create the permissions cache and other necessary data so the system doesn't recognize it as valid.
  2. Mike Tougeron

    Mike Tougeron Well-Known Member

    This time I actually got an error so I'm able to give a bit more info...
    Server Error
    Mysqli statement execute error : Lock wait timeout exceeded; try restarting transaction
    Zend_Db_Statement_Mysqli->_execute() in Zend/Db/Statement.php at line 320
    Zend_Db_Statement->execute() in Zend/Db/Adapter/Abstract.php at line 479
    Zend_Db_Adapter_Abstract->query() in Zend/Db/Adapter/Abstract.php at line 634
    Zend_Db_Adapter_Abstract->update() in XenForo/Model/Permission.php at line 1513
    XenForo_Model_Permission->rebuildPermissionCombination() in XenForo/Model/Permission.php at line 1453
    XenForo_Model_Permission->rebuildPermissionCache() in XenForo/Model/Permission.php at line 1155
    XenForo_Model_Permission->_updatePermissionsForUserCollection() in XenForo/Model/Permission.php at line 1061
    XenForo_Model_Permission->updateContentPermissionsForUserCollection() in XenForo/ControllerAdmin/Permission/Node.php at line 257
    XenForo_ControllerAdmin_Permission_Node->_setPermissionRevokeStatus() in XenForo/ControllerAdmin/Permission/Node.php at line 76
    XenForo_ControllerAdmin_Permission_Node->actionNodeWideRevoke() in XenForo/FrontController.php at line 310
    XenForo_FrontController->dispatch() in XenForo/FrontController.php at line 132
    XenForo_FrontController->run() in xenforo/admin.php at line 72
  3. Jake Bunce

    Jake Bunce XenForo Moderator Staff Member

    Long rebuilds are a known design issue when you have lots of nodes / groups.

    If you have root access to the server then you can edit the my.cnf file (usually at /etc/my.cnf) to increase the value of innodb_lock_wait_timeout. Increasing that limit should avoid this error. If you don't have root access then you need to ask your host or server person to do this.
  4. Mike Tougeron

    Mike Tougeron Well-Known Member

    Yeah, I just didn't see a report for it so I figured I'd make one.

    This is already pretty high for us in production. My work-around for now is to manually update the db via a script. :(
  5. Deepmartini

    Deepmartini Well-Known Member

    Is that because your nodes are mostly "pages"? Maybe another reason why we need better page management and possibly it's own system outside of a "node".
    JVCode likes this.
  6. Mike Tougeron

    Mike Tougeron Well-Known Member

    No, IGN has several thousand boards (18,381 to be exact) for all of the games, movies & tv shows we want boards for.
  7. Mike

    Mike XenForo Developer Staff Member

    The number of "permission combinations" you have (see the table by that name in the DB) has a big effect on this, though the number of nodes obviously applies too.

    To some degree, there's only so much we can do about this, as we need to do rebuilding when this happens. Obviously, if I can manage to speed up the process, I will.

    But really, the better approach (if possible) will be to try to break up the rebuild process (like with templates, etc) so that we can be sure it succeeds. Have some ideas for 1.2 for this.
    simbolo, Walter, Jarod and 2 others like this.
  8. Mike Tougeron

    Mike Tougeron Well-Known Member

    I was thinking about this on the train ride in this morning... What if it only rebuilt the perm cache only when the perms actually changed? Right now it always does it but, at least for us, it is rarely (if at all) different.
  9. Mike

    Mike XenForo Developer Staff Member

    The permissions do change when you create a node - we cache the final permissions for every node. That said, it is naive as to how much it rebuilds (all of it vs just what is changing).
    simbolo and SneakyDave like this.
  10. ashkir

    ashkir Active Member

    Oh! So glad this has been reported already! ^_^ As we're running into this a lot as of late.
  11. Mike

    Mike XenForo Developer Staff Member

    For reference, this is partially fixed in 1.2. There have been some improvements on permission rebuilding in 1.2, though not a fundamental change to the approach. Some superfluous tables/builds have been removed and the full permission build that needs to be done in some situations will automatically be run in parts if needed.

    Further changes probably wouldn't happen before 2.0 unfortunately, though I do have various ideas of things to change.
  12. tyteen4a03

    tyteen4a03 Well-Known Member

    2.0... how bad is the current process?

Share This Page