Closed Activity-triggered automatic jobs may execute concurrently without a documented concurrency contract

zeeb0t

zeeb0t

Well-known member
Licensed customer
Affected version
2.3.11

Summary​


Before you read on: in short, I think this is probably just a documentation improvement for the behavioural difference in execution mode.

When activity-based job triggering is enabled, multiple overlapping job.php requests may execute different automatic jobs concurrently.

The existing atomic claim operations appear to prevent the same xf_job row or the same xf_cron_entry from being claimed simultaneously. They do not, however, serialise automatic queue execution as a whole.

I am not reporting that the same job row or cron entry is being claimed twice. The concern is concurrent execution across different automatic job rows, particularly where those jobs eventually operate on the same logical resource.

Observed implementation​


Each activity-triggered worker calls:

PHP: 
XF\Job\Manager::runQueue(false, ...)

Within the queue:

  • XF\Job\Manager::runJobEntry() attempts to claim an individual xf_job row by moving its trigger_date forward, using its previously read value in the WHERE condition.
  • If another worker has already changed that row, the claim fails and the worker remains able to process another runnable job.
  • XF\Job\Cron::run() similarly claims individual due cron entries by atomically advancing xf_cron_entry.next_run.

These operations protect individual rows. They do not prevent separate workers from successfully claiming and executing different automatic jobs at the same time.

Example race​


  1. Multiple automatic jobs are runnable.
  2. Several rendered pages cause browsers to request job.php at approximately the same time.
  3. Worker A enters runQueue(false, ...) and claims the cron dispatcher job.
  4. Worker B also enters runQueue(false, ...).
  5. Worker B cannot claim the cron dispatcher row because Worker A has already moved its trigger_date.
  6. Worker B proceeds to another runnable automatic job and successfully claims it.
  7. Worker A and Worker B are now executing separate automatic jobs concurrently.

No SQL atomicity failure is required for this to occur. Each individual row claim can operate correctly while distinct jobs execute in parallel.

Observed effect​


A mutation-sensitive maintenance path is protected by a MySQL named lock:

SQL: 
GET_LOCK('exampleMaintenanceLock', 0)

One worker acquires the lock and begins the mutation. Another overlapping worker reaches the same logical maintenance path through a different automatic job and receives 0, indicating that another database connection is already inside the protected section.

Correlated logging can be provided containing the execution source, PHP PID, MySQL connection ID, job class, lock result and timestamps if required.

Documentation​


The current documentation describes activity-based and server-based triggering as alternative mechanisms for initiating background job processing.

I have not found documentation stating that:

  • Multiple activity-triggered automatic workers may execute concurrently.
  • Distinct automatic jobs or cron callbacks may overlap.
  • Add-on jobs sharing a logical mutation must implement their own non-overlap locking.

Expected clarification or behaviour​


Please confirm whether concurrent execution of distinct automatic jobs is intended.

If it is intended, the job and cron documentation should explicitly state that:

  • Automatic job execution is not globally serialised.
  • Different xf_job rows may execute concurrently.
  • Separate cron or job paths may overlap when they access the same logical resource.
  • Add-on developers must make shared mutations idempotent or provide their own logical locking.

If globally serial execution is intended, activity-triggered job.php queue execution would require a runner-level non-overlap mechanism.

Right now, the practical concurrency characteristics of the two trigger modes differ. Activity-based triggering may create multiple overlapping automatic workers through concurrent job.php requests, whereas server-based triggering will typically have a single worker for each scheduler invocation and is therefore less likely to execute jobs concurrently. Overlap remains possible in server-based mode if separate scheduler invocations overlap, but it is not normally exposed to the same level of request-driven concurrency.

In either case, activity-based and server-based triggering should expose a clear and consistent correctness contract, even where their trigger mechanisms and potential number of runner processes differ.
 
Last edited:
Automated triage: As designed

Automatic jobs are protected per job and per cron entry, not by one global lock, so different jobs can run at the same time when several requests trigger the queue. Code that changes a shared resource from more than one job should be idempotent or do its own locking. We've noted that the documentation could say this more clearly.

This is an automated assessment. A member of staff may review and update it.
 
Back
Top Bottom