Partial fix Internal Server Error when moderating multiple posts in very large threads.

TimWF

Active member
We are experiencing internal server errors when we try to delete groups of 3-4 or more posts at a time within large threads (not so helpful screenshot attached :D). The particular threads I have tested/seen this in are in the range of 70,000+ post threads and after looking into the php error log we found that the errors from these all seem to be memory related. On two lines within the Zend Mysqli library:

Code:
[25-Jul-2011 21:15:29] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 128 bytes) in /blahblah/forums/library/Zend/Db/Statement/Mysqli.php on line 304
[25-Jul-2011 21:15:31] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 32 bytes) in /blahblah/forums/library/Zend/Db/Statement/Mysqli.php on line 294
(In total we have somewhere around 10K errors of these two types over the past few days, though they are not all related to the delete functionality certainly. We aren't sure where they are also being triggered.)
We have also seen some reports that alerts for the thread might not be working in addition, but I haven't seen any errors that I can see for certain are related to this as of yet.
 

Attachments

  • internal server error.webp
    internal server error.webp
    19.1 KB · Views: 44
Very annoyingly, I'm not sure what we can deduce from that without a full backtrace - that's just erroring within data from a query. I have a possible guess of where it is, but I'm not positive yet.
 
Thanks Mike, glad to hear you are looking into. If there is any information or tests that we could run that could help you out let me know and we will do our best to accommodate.
 
We are also getting this on the same line on a thread with lots of replies:

PHP:
[11-Aug-2011 20:38:33] PHP Fatal error:  Out of memory (allocated 45350912) (tried to allocate 32 bytes) in /home/xxxxxx/public_html/library/Zend/Db/Statement/Mysqli.php on line 294
[12-Aug-2011 11:02:18] PHP Fatal error:  Out of memory (allocated 45350912) (tried to allocate 32 bytes) in /home/xxxxxx/public_html/library/Zend/Db/Statement/Mysqli.php on line 294

I've checked our Apache and PHP memory limits and they are far in excess of this limit.
 
Edward, your server is limited to allocating ~45MB for some reason. Your memory limits aren't coming into it (unless of course that is a limit from somewhere). I'd try to look into that or simply create "part 2" threads.

I have a partial fix for this, that at least roughly halves the memory usage from before. I am looking into a different approach to fixing it for 1.1 as it involves API changes so I can't do that for 1.0.

In library/XenForo/Model/Post.php, look for:
Code:
        return $this->fetchAllKeyed('
            SELECT post_id, thread_id, user_id, username,
                post_date, ip_id, message_state, position, attach_count, likes
                ' . ($includeMessage ? ', message' : '') . '
            FROM xf_post
            WHERE thread_id = ?
            ORDER BY position ASC, post_date ASC
        ', 'post_id', $threadId);

Replace with:
Code:
        return $this->fetchAllKeyed('
            SELECT post_id, user_id, message_state, likes
                ' . ($includeMessage ? ', message' : '') . '
            FROM xf_post
            WHERE thread_id = ?
            ORDER BY position ASC, post_date ASC
        ', 'post_id', $threadId);
This limits the returned data (only used here) to the absolute base amount of what's needed. (In theory this could break an add-on too, but hopefully not.)
 
Same error appears with that code used also, here's the memory details:

PHP:
bash-3.2# php -i | grep memory_
memory_limit => 128M => 128M

We do not have 45MB specified anywhere (Apache is set to use 128MB) and have no custom php.ini's - we use the servers default php.ini. MySQL is running with the default cPanel my.cnf (no customisations).

Thanks for looking into this.
 
I have a partial fix for this, that at least roughly halves the memory usage from before. I am looking into a different approach to fixing it for 1.1 as it involves API changes so I can't do that for 1.0.

Mike, just wanted to check back in and see if the possible different approach for 1.1 made it into 1.1? Thanks!
 
Thanks for the heads up Mike. Any estimate on when we might be looking at?

Additionally, we just very recently did start using the suggested alteration of Posts.php and thus far haven't run into any conflicts with any add-ons. Thanks.
 
I think that version of the code might've actually had some issues in it -- at least in 1.1, so the official version probably grabs a few other bits.

Ideally would look to due a full fix in 1.2.
 
Again, thanks for the heads up on that as well. We are still on 1.0, though we are looking to make the move to 1.1 I can't speak for how that change works in 1.1. Sounds like it would not quite work though.

Is the 1.1 version of that query as optimized as it can be by default?
 
upload_2013-10-25_14-53-5.webp

I have alot of users awaiting approval.

I get this error when attempting to access the list.

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 16 bytes) in /home/content/html/community/library/Zend/Db/Statement/Mysqli.php on line 294

upload_2013-10-25_14-54-18.webp

FYI.
 
Top Bottom