Resource icon

GoodForNothing Shell Defer 1.0.4

No permission to download
The deferred job runner is atomic within a particular job. The process that successfully updates the job is the one that runs it; others that have tried to grab that job in the interim will have their update fail so they will skip the job. So in that regard, another lock isn't explicitly needed. Multiple simultaneous runners shouldn't be an issue unless it starts overloading the server (if you have multiple cores available then multiple jobs could run simultaneously with minimal slowdown).

That is what I've thought also. Thanks for clarification.

I see no need for a defer lock if you run it from the shell. But of course it does not hurt either being more complicated than necessary. ;)
 
Installing this on AVForums generated 576 errors in 3 minutes like this :
ErrorException: Undefined property: stdClass::$_type - library/XenES/Search/SourceHandler/ElasticSearch.php:870
Generated By: Unknown Account, 3 minutes ago
Stack Trace
#0 /home/sites/avforums/public_html/library/XenES/Search/SourceHandler/ElasticSearch.php(870): XenForo_Application::handlePhpError(8, 'Undefined prope...', '/home/sites/avf...', 870, Array)
#1 /home/sites/avforums/public_html/library/XenES/Model/Elasticsearch.php(264): XenES_Search_SourceHandler_ElasticSearch->isIndexSuccessful(Object(stdClass))
#2 /home/sites/avforums/public_html/library/XenES/Model/Elasticsearch.php(315): XenES_Model_Elasticsearch->reattemptFailedIndex('index', 'post', 21644366, Array, 5)
#3 /home/sites/avforums/public_html/library/XenES/CronEntry/EsReindex.php(19): XenES_Model_Elasticsearch->runFailedReindexer()
#4 [internal function]: XenES_CronEntry_EsReindex::reindex(Array)
#5 /home/sites/avforums/public_html/library/XenForo/Model/Cron.php(356): call_user_func(Array, Array)
#6 /home/sites/avforums/public_html/library/XenForo/Deferred/Cron.php(24): XenForo_Model_Cron->runEntry(Array)
#7 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(294): XenForo_Deferred_Cron->execute(Array, Array, 7.9999978542328, '')
#8 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(428): XenForo_Model_Deferred->runDeferred(Array, 7.9999978542328, '', false)
#9 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(373): XenForo_Model_Deferred->_runInternal(Array, NULL, '', false)
#10 /home/sites/avforums/public_html/deferred.php(23): XenForo_Model_Deferred->run(false)
#11 {main}
Request State
array(3) {
["url"] => string(37) "https://www.avforums.com/deferred.php"
["_GET"] => array(0) {
}
["_POST"] => array(4) {
["_xfRequestUri"] => string(70) "/threads/what-is-it-guess-the-close-up-thread-part-17.1725418/page-122"
["_xfNoRedirect"] => string(1) "1"
["_xfToken"] => string(8) "********"
["_xfResponseType"] => string(4) "json"
}
}
 
Installing this on AVForums generated 576 errors in 3 minutes like this :

@Stuart Wright : You do not need an add-on for this task.
Just remove the deferred.php trigger from PAGE_CONTAINER and add a line like this one to your /etc/crontab at one of your servers:
Code:
* * * * * apache /usr/bin/php /var/www/****/deferred.php > /dev/null 2>&1

If your web server user is not named "apache" change that. Also you need to adapt the path to deferred.php

This setup is sufficient and runs very smoothly at even very large forums (which are the only ones needing such an external trigger.
 
@Stuart Wright : You do not need an add-on for this task.
Just remove the deferred.php trigger from PAGE_CONTAINER and add a line like this one to your /etc/crontab at one of your servers:
Code:
* * * * * apache /usr/bin/php /var/www/****/deferred.php > /dev/null 2>&1

If your web server user is not named "apache" change that. Also you need to adapt the path to deferred.php

This setup is sufficient and runs very smoothly at even very large forums (which are the only ones needing such an external trigger.
I got the path correct when I edited the cron and Tim @ Nimbus confirmed it was ok.
*/2 * * * * /usr/bin/php -q /home/sites/avforums/public_html/cli/defer.php 2>&1
So the addon does nothing extra that I need?
What does it do that would cause all these server errors?
 
The add-on does nothing extra you need. In fact it makes things more complicated and more prone to failures.

Code:
*/2 * * * * /usr/bin/php -q /home/sites/avforums/public_html/cli/defer.php 2>&1

If you put this line in /etc/crontab it would not work, because there is no user specified. If you put this line in a cron of a user, you should NOT run it from the root user, but from the web server users cron!

Also the original file in XenForo is named "deferred.php" and NOT "defer.php"!

A forum your size should trigger deferred.php each minute and not every second minute only.
 
Last edited:
The add-on does nothing extra you need. In fact it makes things more complicated and more prone to failures.

Code:
*/2 * * * * /usr/bin/php -q /home/sites/avforums/public_html/cli/defer.php 2>&1

If you put this line in /etc/crontab it would not work, because there is no user specified. If you put this line in a cron of a user, you should NOT run it from the root user, but from the web server users cron!

Also the file in XenForo is named "deferred.php" and NOT "defer.php"!
Whoosh over my head. I do all the stuff in the cron after logging in as root. Never done it any other way.
I've uninstalled the addon, put the deferred.php cron job in there and removed the deferred thingy from the page_container template. Thanks for the help.
 
Whoosh over my head. I do all the stuff in the cron after logging in as root. Never done it any other way.
I've uninstalled the addon, put the deferred.php cron job in there and removed the deferred thingy from the page_container template. Thanks for the help.

This looks to me like you run deferred.php from the root user now. I would NOT recommend to do this!
It should be triggered from the web server user. Please talk with Tim @ Nimbus about how to do this.

If you run it from root it is a) a security risk and b) a huge problem if a directory or file will be created by the root user and not the web server user during a deferred task!
 
Last edited:
I made this add-on for two reason:
  • for smaller boards, it means the deferred process will take place even if there are no visitor for some time.
  • for bigger boards, it will help reduce the resource usage as there will be no extra TCP calls that the web server has to process.
The error Stuart posted might not be related to this add-on as it does not do anything to XenForo's original deferred.php which was traced in the log :)
But yes, you should run the cron as the user who has the permission for the site.

What does the -q after php do?
IIRC the system will queue the process if there's no threads open... I might be wrong :)
 
Actually, the q flag is a quiet option, to not output anything. Not sure if it's needed in this instance, but most php scripts I've seen executed via cron include the flag
 
I set up a xenforo cron to run at 3:30 AM (SneakyDave's backup add-on).
And today I checked my backup and it says it was made at 5:55 AM (last amended time for that file, also the backup log says the same).
This is my cron log (var/log/cron) from that time period.

Code:
Apr 23 03:00:01 server CROND[22113]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:00:01 server CROND[22114]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 23 03:01:01 server CROND[22156]: (root) CMD (run-parts /etc/cron.hourly)
Apr 23 03:01:01 server run-parts(/etc/cron.hourly)[22156]: starting 0anacron
Apr 23 03:01:01 server anacron[22165]: Anacron started on 2015-04-23
Apr 23 03:01:01 server run-parts(/etc/cron.hourly)[22167]: finished 0anacron
Apr 23 03:01:01 server anacron[22165]: Will run job `cron.daily' in 41 min.
Apr 23 03:01:01 server anacron[22165]: Jobs will be executed sequentially
Apr 23 03:02:01 server CROND[22173]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:04:01 server CROND[22185]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:06:01 server CROND[22201]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:08:01 server CROND[22215]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:10:01 server CROND[22230]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:10:01 server CROND[22229]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 23 03:10:01 server CROND[22231]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 23 03:12:01 server CROND[22277]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:14:01 server CROND[22289]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:16:01 server CROND[22305]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:18:01 server CROND[22319]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:20:01 server CROND[22334]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:20:01 server CROND[22335]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 23 03:20:01 server CROND[22336]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 23 03:22:01 server CROND[22384]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:24:01 server CROND[22396]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:26:01 server CROND[22412]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:28:01 server CROND[22426]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:30:01 server CROND[22440]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 23 03:30:01 server CROND[22443]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:30:01 server CROND[22445]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 23 03:32:01 server CROND[22488]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:34:01 server CROND[22500]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:36:01 server CROND[22516]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:38:01 server CROND[22532]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:40:01 server CROND[22547]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 23 03:40:01 server CROND[22546]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 23 03:40:01 server CROND[22548]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:42:01 server anacron[22165]: Job `cron.daily' started
Apr 23 03:42:01 server run-parts(/etc/cron.daily)[22593]: starting 0yum.cron
Apr 23 03:42:01 server CROND[22608]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:44:01 server CROND[22620]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:46:01 server CROND[22636]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:48:01 server CROND[22650]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:50:01 server CROND[22664]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:50:01 server CROND[22665]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 23 03:50:01 server CROND[22667]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 23 03:52:01 server CROND[22713]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:54:01 server CROND[22725]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:56:01 server CROND[22754]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22813]: finished 0yum.cron
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22593]: starting diskalert
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22844]: finished diskalert
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22593]: starting freshclam
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22851]: finished freshclam
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22593]: starting logrotate
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22858]: finished logrotate
Apr 23 03:56:32 server run-parts(/etc/cron.daily)[22593]: starting maldet
Apr 23 03:58:01 server CROND[22870]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 03:59:12 server run-parts(/etc/cron.daily)[23036]: finished maldet
Apr 23 03:59:12 server run-parts(/etc/cron.daily)[22593]: starting mlocate.cron
Apr 23 03:59:13 server run-parts(/etc/cron.daily)[23104]: finished mlocate.cron
Apr 23 03:59:13 server anacron[22165]: Job `cron.daily' terminated (mailing output)
Apr 23 03:59:13 server anacron[22165]: Normal exit (1 job run)
Apr 23 04:00:01 server CROND[23141]: (root) CMD (/usr/bin/cminfo_updater)
Apr 23 04:00:01 server CROND[23140]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Apr 23 04:00:01 server CROND[23143]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 04:00:01 server CROND[23142]: (root) CMD (/usr/local/maldetect/maldet --mkpubpaths >> /dev/null 2>&1)
Apr 23 04:01:01 server CROND[23186]: (root) CMD (run-parts /etc/cron.hourly)
Apr 23 04:01:01 server run-parts(/etc/cron.hourly)[23186]: starting 0anacron
Apr 23 04:01:01 server run-parts(/etc/cron.hourly)[23195]: finished 0anacron
Apr 23 04:02:01 server CROND[23201]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 04:04:01 server CROND[23213]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 04:06:01 server CROND[23229]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)
Apr 23 04:08:01 server CROND[23243]: (root) CMD (php -q /home/nginx/domains/xxx.com/public/cli/defer.php  2>&1)

Do you know what the problem is maybe? The backup should have been done 2:30 hours earlier. But it didn't. The cron seems to work. The whole day for every second minute I get that entry. But still the xenforo cron didn't run at that time it supposed to be.
 
Last edited:
Can someone explain to me why you would want to run deferred via a cron? I have a pretty busy forum and like Stuart have no shortage of users who will regularly trigger the deferred script in the normal way without a cron
 
Can someone explain to me why you would want to run deferred via a cron? I have a pretty busy forum and like Stuart have no shortage of users who will regularly trigger the deferred script in the normal way without a cron
1) Well you have a busy forum, but not everyone does.
2) Is your forum also busy at nights/mornings like 3-5AM in the morning? Because I want to backup my forum when it is not busy, so when you expect the least visitors. So you want that the xenforo cron runs at that time it supposed to run. Like me, if you want to close your board for backups, it is a must that the xenforo cron runs exactly at that time it should.

For example I planned to close my board at 3:25 AM and open it at 3:35 AM again via a cron. So every day my board will be closed at those times. Because I want to backup my stuff in those 10 minutes. But the problem is now, because you have no visitors, the xenforo cron doesn't run, and the forum gets closed a lot later AND your backups get backed up a lot later. Not in a scheduled timeframe.

For that you need this trigger.
 
I've explained already I have no shortage of users, many requests per minute day and night.
I dont understand your backup argument because I dont think that has anything to do with the deferred process.
 
Don't be such self-centered. This is not for people who have a busy forum. Just ignore this, if you are in that category.
The deferred process -> activates Xenforo crons (-> which activates my backup process for example or any other crons you want to have run exactly at a specific time)
 
Dont be so rude. I'm not slating this mod or the method so climb down off your ivory tower and put a different lens in your monacle.

I know I dont NEED this, my question was largely aimed a Stuart, who also has a busy board, but who is also, clearly cron'ing up the deferred process. You can simply ignore this sbj.
 
I know I dont NEED this,
BUT
Can someone explain to me why you would want to run deferred via a cron?
I tried to explain and your response was:
I've explained already I have no shortage of users, many requests per minute day and night.
???
my question was largely aimed a Stuart
No, you asked if someone can explain to you. Should we be able to read minds now?

At the end, I tried to help out and I am the rude one. Sure...
 
Hey you started with the insults and no need for mind reading is there? I said I had a busy forum. I know about the benefits to quiet websites but I'm seeing what other benefits exist.
@Stuart Wright can you add to this?
 
okay guys calm down... lol :P
To answer @Rob 's question:
I made this add-on for two reason:
  • for smaller boards, it means the deferred process will take place even if there are no visitor for some time.
  • for bigger boards, it will help reduce the resource usage as there will be no extra TCP calls that the web server has to process.
Though I do know that for such boards you will be using a more or less powerful server but you know... :D
 
  • Like
Reactions: Rob
Top Bottom