CLI Job Runner

CLI Job Runner 1.3.1

No permission to download

Sim

Well-known member
Sim updated CLI Job Runner with a new update entry:

v1.2.0

Thanks to @Xon for identifying the bug fix where AJAX JSON responses would still trigger job.php; and for making suggestions on avoiding unnecessary database queries and prevent running cron jobs unless triggered via our CLI job runner
  • changes: disable calculations for job auto run time to avoid unnecessary database queries
  • bug fix: disable AJAX auto job runner from triggering from AJAX JSON responses
  • changes: don't allow cron tasks to execute unless...
Read the rest of this update entry...
 

Sim

Well-known member
@Sim there are a couple places you need to override to stop job.php from being polled.

PHP:
class Manager extends XFCP_Manager
{
  public $allowCron = false;

  public function setAllowCron($value)
  {
      $this->allowCron = $value;
  }

  public function scheduleRunTimeUpdate()
  {
      if (empty(\XF::options()->svRunCronFromCron))
      {
          parent::scheduleRunTimeUpdate();
      }
  }

  public function getRunnable($manual)
  {
      if ($manual || $this->allowCron)
      {
          return parent::getRunnable($manual);
      }

      if (empty(\XF::options()->svRunCronFromCron))
      {
          return parent::getRunnable($manual);
      }

      return $this->db->fetchAll("
   SELECT *
   FROM xf_job
   WHERE trigger_date <= ?
      AND manual_execute = ?
       AND unique_key != 'cron'
   ORDER BY trigger_date
   LIMIT 1000
", [\XF::$time, $manual ? 1 : 0]);
  }
}
You want to extend \XF\Job\Manager::scheduleRunTimeUpdate to catch autoJobRun being updated to reduce unneeded DB writes.

Extending \XF\Job\Manager::getRunnable is needed because if a job was queued in a request, then XF sets various attributes in addDefaultJsonParams based on internal state (and not the template autoRun variable!) which means the cron job could be evaluated when job.php executes.

Then update the CLI runner command to call setAllowCron(true).

As for why you would want to block the cron job from being executed via job.php;
View attachment 212841
Before 12:00, standard xf web-based cron with autoRun being removed like your add-on does.
After 12:00, the above code changes in a private add-on is deployed.

This is the classic "thundering herd" problem, where a large number of requests wake up at once, all go to get a lock and then go back to sleep after most fail to get anything done.
Thanks @Xon - I've just release v1.2.0 which addresses the issues you identified.

I had indeed missed the fact that AJAX JSON responses could cause job.php to be called. Pretty simple fix to disable that - although calling job.php is still required for AutoBlocking jobs - since they change the UI and actually do need to be executed immediately (currently only used by the Approval Queue).

Good call on preventing the registry updates when calculating next run times - they are indeed unnecessary when we are using our cron triggered CLI job runner.

Finally - although the previous changes should in theory prevent cron tasks from running when other jobs are triggered (except via the Approval Queue), it's probably a safer approach to be more explicit and only allow it when triggered via the CLI job runner in any case - thanks for the suggestion!
 
  • Like
Reactions: Xon

Skyrider

Active member
I'm curious. Is there anyway I can see the debug output/input of the cronjob? Sometimes a specific cron job takes a long while finish, so it would be nice to know to see some kind of debug log as to see where it is "hanging". Without the quiet command line, it'll merely show "No more runnable jobs pending", but I assume that's what the quiet line hides.
 

Sim

Well-known member
I'm curious. Is there anyway I can see the debug output/input of the cronjob? Sometimes a specific cron job takes a long while finish, so it would be nice to know to see some kind of debug log as to see where it is "hanging". Without the quiet command line, it'll merely show "No more runnable jobs pending", but I assume that's what the quiet line hides.
@Skyrider - take a look at the new release v1.3.1 - I've added the ability to show some basic debugging information on the console when xf:run-jobs is run in Verbose (-v), Very Verbose (-vv) or Debug (-vvv) modes.

You can also add logging code to your custom Jobs and Cron tasks to be displayed on the console for debugging purposes - even on a production machine. See instructions at the bottom of the addon overview page.
 

Xon

Well-known member
For the record; using systemd timers instead of cron;

Timer; /etc/systemd/system/xf-cron.timer
Code:
[Unit]
Description=Timer trigger for xf-cron service
Wants=network-online.target

[Timer]
OnBootSec=1min
OnUnitActiveSec=20sec

[Install]
WantedBy=multi-user.target
Service (that the timer triggers); /etc/systemd/system/xf-cron.service
Code:
[Unit]
Description=xf-cron service

[Service]
Type=oneshot
User=www-data
Group=www-data
ExecStart=/usr/bin/php -q /path/to/your/forum/root/cmd.php --quiet xf:run-jobs
Change www-data to the user your site runs as, and /path/to/your/forum/root/ as required

Configure system;
Code:
systemd daemon-reload
systemd enable xf-cron.timer
systemd start xf-cron.timer
 

Sim

Well-known member
Push Notifications will be process via this addon/cron?
This addon just processes any and all jobs queued by the system - so if push notifications are sent via a job, then yes it will.

It doesn't add or change any functionality beyond disabling the browser based job runner and replacing it with a CLI based job runner that does exactly the same thing that the browser one did - run all the queued jobs.
 
  • Like
Reactions: rdn

rdn

Well-known member
I hope there's an option to force Push Notification be run always via this addon/cli.
Thanks Sim and Xon.
 
Top