XF 1.2 Questions About Deferred

digitalpoint

Well-known member
Before 1.2, I disabled cron.php from being triggered by browser activity, and instead set up a crontab job on one of my servers to fire it off every 1 minute. That worked fine... but with 1.2, I decided to go back to how it works by default.

Is It Atomic?

After checking some of my web logs, I'm wondering if that was a good idea or not... deferred.php is being called 5-10 times per second (even Googlebot is triggering it to happen)...

For example, it was kicked off 28 times in this 4 second window (that's not unique, it's like that always).

Should it really be called that often?
Code:
66.249.74.17 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/internet-explorer-8-better-than-firefox.1929807/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
176.35.183.18 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/domain-name-expired-how-long-until-available.10918/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.59.8 (KHTML, like Gecko) Version/5.1.9 Safari/534.59.8"
122.161.247.237 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/ecommerce.115/" "Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20100101 Firefox/21.0"
66.249.74.27 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-get-rank-a-web-site-on-search-engine.1893435/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
108.199.254.160 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.45 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/howdy-howdy-hi.415148/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.6 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/offering-work-for-blog-posters.144610/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.45 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/obama-approval-rating-falling.1244046/page-79" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.27 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/making-money-help.1968482/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
137.222.52.4 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/havnt-havent.1152572/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.17 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-ruin-a-web-site.860200/page-4" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
80.239.243.18 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/which-site-can-i-exchange-my-paypal-to-webmoney.1108869/" "Opera/9.80 (BlackBerry; Opera Mini/7.1.32723/30.3593; U; en) Presto/2.8.119 Version/11.10"
66.249.74.6 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/need-unique-myspace-layouts.128868/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
108.199.254.160 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/le-software-development.1364774/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/need-posters-on-webmasters-forum.1321587/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/new-free-submission-directory-no-recip.872235/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.17 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/paypal-cheque-not-received-yet-what-should-i-do-suggestion-please.2366498/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
81.140.26.165 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://www.digitalpoint.com/" "Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0"
60.225.40.112 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/what-does-x64-and-x86-mean.674631/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.45 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/human-traffic-includes-detailed-statistics-country-targetting-0-30-1000-visits.2499188/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
108.199.254.160 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
113.179.29.199 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/best-language-to-start-for-web-development.2637702/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0"
66.249.74.17 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/increasing-your-adsensing-earnings-200.1289098/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.6 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/huge-traffic-mobile-website-for-sale.1440274/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
193.239.73.222 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/conversations/add?to=espmartin" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.7 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/is-this-a-website-or-blog.2004821/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.28 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-recover-penalized-websites.2575169/page-2" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Weirdness On Sub-domains

I also noticed a route of "deferred" (without the .php) is being called, but only if the user is on a sub-domain. That's not a valid route even on the primary domain, and it's a GET (not a POST). So it's not just an issue of it not using the HTML base properly.

For example:

Code:
66.208.237.49 - - [07/Aug/2013:12:20:16 -0700] "GET /deferred HTTP/1.1" 404 6631 "https://tools.digitalpoint.com/suggestion" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
69.180.14.74 - - [07/Aug/2013:12:20:49 -0700] "GET /deferred HTTP/1.1" 404 6630 "https://tools.digitalpoint.com/tracker" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.218 - - [07/Aug/2013:12:26:37 -0700] "GET /deferred HTTP/1.1" 404 6626 "https://tools.digitalpoint.com/cookie-search?domain=stumpchuck.com" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

I've tried to find where that's coming from, but have been unable to find it (it never requests that URL when on the main sub-domain). Any ideas?
 
It is atomic (same as cron.php, which probably would've run into the same issues). You can probably keep the cron.php-style system you had, just make sure that you let the deferreds run for longer than 10 seconds at a time (config.php option).

I'm really not sure about the deferred route. You have all manner of changes in place, so I'm not sure what's going on there.
 
So there is nothing that is deferred that is expected to run instantly with it? I'll set up deferred.php to run once a minute via crontab and see if I break anything.

Will go digging on the sub-domain thing. Definitely nothing I added, but determined to figure that one out.
 
Back
Top Bottom