XF 1.2 Questions About Deferred

digitalpoint

digitalpoint

Well-known member
Before 1.2, I disabled cron.php from being triggered by browser activity, and instead set up a crontab job on one of my servers to fire it off every 1 minute. That worked fine... but with 1.2, I decided to go back to how it works by default.

Is It Atomic?

After checking some of my web logs, I'm wondering if that was a good idea or not... deferred.php is being called 5-10 times per second (even Googlebot is triggering it to happen)...

For example, it was kicked off 28 times in this 4 second window (that's not unique, it's like that always).

Should it really be called that often?
Code: 
66.249.74.17 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/internet-explorer-8-better-than-firefox.1929807/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
176.35.183.18 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/domain-name-expired-how-long-until-available.10918/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.59.8 (KHTML, like Gecko) Version/5.1.9 Safari/534.59.8"
122.161.247.237 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/ecommerce.115/" "Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20100101 Firefox/21.0"
66.249.74.27 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-get-rank-a-web-site-on-search-engine.1893435/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
108.199.254.160 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.45 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/howdy-howdy-hi.415148/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.6 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/offering-work-for-blog-posters.144610/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.45 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/obama-approval-rating-falling.1244046/page-79" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.27 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/making-money-help.1968482/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
137.222.52.4 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/havnt-havent.1152572/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.17 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-ruin-a-web-site.860200/page-4" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
80.239.243.18 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/which-site-can-i-exchange-my-paypal-to-webmoney.1108869/" "Opera/9.80 (BlackBerry; Opera Mini/7.1.32723/30.3593; U; en) Presto/2.8.119 Version/11.10"
66.249.74.6 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/need-unique-myspace-layouts.128868/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
108.199.254.160 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/le-software-development.1364774/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/need-posters-on-webmasters-forum.1321587/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/new-free-submission-directory-no-recip.872235/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.17 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/paypal-cheque-not-received-yet-what-should-i-do-suggestion-please.2366498/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
81.140.26.165 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://www.digitalpoint.com/" "Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0"
60.225.40.112 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/what-does-x64-and-x86-mean.674631/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.45 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/human-traffic-includes-detailed-statistics-country-targetting-0-30-1000-visits.2499188/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
108.199.254.160 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
113.179.29.199 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/best-language-to-start-for-web-development.2637702/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0"
66.249.74.17 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/increasing-your-adsensing-earnings-200.1289098/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.6 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/huge-traffic-mobile-website-for-sale.1440274/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
193.239.73.222 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/conversations/add?to=espmartin" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.7 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/is-this-a-website-or-blog.2004821/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.28 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-recover-penalized-websites.2575169/page-2" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Weirdness On Sub-domains

I also noticed a route of "deferred" (without the .php) is being called, but only if the user is on a sub-domain. That's not a valid route even on the primary domain, and it's a GET (not a POST). So it's not just an issue of it not using the HTML base properly.

For example:

Code: 
66.208.237.49 - - [07/Aug/2013:12:20:16 -0700] "GET /deferred HTTP/1.1" 404 6631 "https://tools.digitalpoint.com/suggestion" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
69.180.14.74 - - [07/Aug/2013:12:20:49 -0700] "GET /deferred HTTP/1.1" 404 6630 "https://tools.digitalpoint.com/tracker" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
66.249.74.218 - - [07/Aug/2013:12:26:37 -0700] "GET /deferred HTTP/1.1" 404 6626 "https://tools.digitalpoint.com/cookie-search?domain=stumpchuck.com" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

I've tried to find where that's coming from, but have been unable to find it (it never requests that URL when on the main sub-domain). Any ideas?
 
Sort by date Sort by votes
It is atomic (same as cron.php, which probably would've run into the same issues). You can probably keep the cron.php-style system you had, just make sure that you let the deferreds run for longer than 10 seconds at a time (config.php option).

I'm really not sure about the deferred route. You have all manner of changes in place, so I'm not sure what's going on there.
 
Upvote 0 Downvote
So there is nothing that is deferred that is expected to run instantly with it? I'll set up deferred.php to run once a minute via crontab and see if I break anything.

Will go digging on the sub-domain thing. Definitely nothing I added, but determined to figure that one out.
 
Upvote 0 Downvote
You must log in or register to reply here.

Similar threads

Jawsh
  • Suggestion Suggestion
Lack of interest Throttle reactions for registered members
Replies
1
Views
813
Chris D
Chris D
S
  • Question Question
XF 2.0 company closed because my site is attacking
Replies
10
Views
897
Joe Kuhn
Joe Kuhn
B
  • Question Question
XF 2.0 Possible NGINX config issue?
Replies
3
Views
4K
bostoneo
B
RippC
edit avatar results in browser "cookies required" error and http 400 server response. AWS involved why?
Replies
11
Views
1K
rdn
R
Marcus
  • Question Question
XF 2.1 css.php not returning text for most first views
Replies
1
Views
585
Marcus
Marcus
Top Bottom