XF 1.3 Cron job issues

[caleb89taylor]

I've posted some related issues to this before that ended up being resolved or partially resolved, but there seems to be an underlying issue of at least some of our cron jobs not running properly.

Here's are two big issues:
1. The cron job that updates view counters doesn't seem to work. Our xf_thread_view table never gets emptied unless we run the job manually. I'm not sure how to tell if the "view" counts are getting updated but the memory table definitely isn't getting cleared.
2. Some users aren't getting trophies points awarded. I've mentioned this in a post before that I thought might be a non-issue as I myself got trophies points awarded. However, it still seems to be a problem. Active users are hitting milestones and not getting trophies. We are talking they haven't received trophy points for months. One user I saw has 0 trophy points despite having 2100 posts and we give several trophies for far few posts than that.

I'm worried that there are more issues that we don't know about. Some users have complained about seeing all 0's for forum statistics as an example. I would like to once and for all knock this bug out and hopefully resolve issues we've been having.

My big questions:
1. I assume all cron jobs are not failing. This would seem to cause a lot of issues.
2. If it's not the cron jobs, then it could be custom code we've written. I know at least for the thread view cron job we have custom classes that have overwritten methods that handle the updating of xf_thread and truncating of the table xf_thread_view (which seems like an obvious culprit except for the odd fact that running manually works).

What steps should I take to debug to determine if it's cron job or custom code? I've started by checking the xf_cron_entry table and for one of the cron jobs that I know has issues, XenForo_CronEntry_Views::runViewUpdate, does have a next_run field in the future.

Any help would be greatly appreciated.
Caleb

 

[Jeremy]

There were some cron improvements in later versions. Are you able to update?

 

[caleb89taylor]

Upgrading isn't an option right now. We mostly just want to stabilize the current version we are on as it still has a lot of bugs (mostly from our custom code and themes).

 

[Paul B]

What steps should I take to debug to determine if it's cron job or custom code?

The first step would be to remove all custom code and disable all add-ons.

Then ensure all default cron tasks are running normally before reapplying each bit of custom code and add-on and checking again.

Edit: I've just realised which site this is for so that approach is likely not going to be very feasible.

 

[Mike]

I would probably add logging code to the cron runner (XenForo_Deferred_Cron) that logs when each task is started and when it completes. If the completion entry doesn't get logged, you may find that something is causing PHP to stop. You may want to use something like PHP's error_log to ensure any issues are properly logged to a file (in case it's a memory issue/fatal error, which is quite hard for a PHP app to consistently log).

 

[caleb89taylor]

Edit: I've just realised which site this is for so that approach is likely not going to be very feasible.

Yeah, we wouldn't want to do that unless it's a last resort.

I would probably add logging code to the cron runner (XenForo_Deferred_Cron) that logs when each task is started and when it completes. If the completion entry doesn't get logged, you may find that something is causing PHP to stop. You may want to use something like PHP's error_log to ensure any issues are properly logged to a file (in case it's a memory issue/fatal error, which is quite hard for a PHP app to consistently log).

I will try this. It could be a memory thing since our xf_thread_view table is pretty big given our traffic. There is also the issue that the default method for updating thread views looks like this:

    public function updateThreadViews()
    {
            $db = $this->_getDb();

            $db->query('
                    UPDATE xf_thread
                    INNER JOIN (
                            SELECT thread_id, COUNT(*) AS total
                            FROM xf_thread_view
                            GROUP BY thread_id
                    ) AS xf_tv ON (xf_tv.thread_id = xf_thread.thread_id)
                    SET xf_thread.view_count = xf_thread.view_count + xf_tv.total
            ');

            $db->query('TRUNCATE TABLE xf_thread_view');
    }

and our custom method (written back in the day for xf 1.1) looks like this:

    public function updateThreadViews()
    {
        // Read from the slave.
        $updates = XenForo_Application::get('dbSlave')->fetchPairs('
            SELECT thread_id, COUNT(*)
            FROM xf_thread_view
            GROUP BY thread_id
        ');

        // Update on the master
        $db = $this->_getDb();
        $db->query('TRUNCATE TABLE xf_thread_view');

        foreach ($updates AS $threadId => $views)
        {
            $db->query('
                UPDATE xf_thread SET
                    view_count = view_count + ?
                WHERE thread_id = ?
            ', array($views, $threadId));
        }
    }

So you can see there is some code where we read from the slave to get the updates and then insert into master. One thought was that the section:

$updates = XenForo_Application::get('dbSlave')->fetchPairs('
            SELECT thread_id, COUNT(*)
            FROM xf_thread_view
            GROUP BY thread_id
        ');

could fail because maybe the XenForo_Application::get('dbSlave') fails on an asynchronous call because it's not bootstrapped in the same way and the application registry isn't available or something odd like that.

 

[caleb89taylor]

Update: Replacing our custom "public function updateThreadViews()" with the default Xenforo one seems to have fixed the issue. I'll keep an eye to make sure it keeps working as intended. Now working in investigating the trophy points issue. I'll see if we have any custom code that could be the culprit for that..

 

[caleb89taylor]

It doesn't appear to be a widespread failure of trophies being awarded. One mod has suggested it's possibly users that have had things like post count manually modified but I don't think that's the case for all users. Still investigating.

 

[Mike]

I suspect that the view issue was a memory issue -- you're running into PHP memory limits in the old version (which is based on the old standard code). The new version is entirely in MySQL so doesn't have all that overhead.

In terms of trophies, can you confirm that user last activity times are being updated? You can see this in the ACP user edit page. (The value is only updated once an hour, but it should be at least reasonably accurate.)

Trophies are not only awarded via cron though; they're also awarded when a session is created for a user (and then for users active in the last few hours via cron).

 

[Resolution]

I'm running Xenforo 1.4.5. I have setup a cron job which works both manually and automatically but am getting strange results.

The cron job is set to copy a file (approx 1mb) from an external server to the local ftp. The original settings are:

Day of the week: Any
Run at hours: 0
Run at minutes: 1
Allow cron entry to run automatically when scheduled: Yes

The time stamp on the file that is copied to my server varies:
09:22
09:35 log says: [24/Jul/2015:09:34:34 +0100]

Changing the cron settings to:
Day of the week: Any
Run at hours: 1
Run at minutes: 2
Allow cron entry to run automatically when scheduled: Yes

08:37 log says: [25/Jul/2015:10:51:25 +0100]

So, my question is that the timestamp on the file does not necessarily match the log file. Changing the cron job run an hour later actually caused it to run an hour earlier according to the timestamp.

Does anyone have any ideas about why the time settings in the XenForo cron job is not correct, I need to be able to tell my clients a time of day when the job is run and the new file will be available?

 

[Mike]

Cron jobs in XF can't be considered to be reliable down to the specific time they run. Effectively, the run time is the first potential run time. The crons are triggered by user activity on the forum. If no one visits, they won't be triggered. If you need specific run time support, you need to use the system cron.

 

[Resolution]

Thank you for the response. I'm building an add on which is why the cron is needed. Is there a particular way I can call the same Xen function I am calling in the Xen cron as per the system cron? I have managed to successfully run .php code from the system cron but am unsure how to call MVC functions?