1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

XF 1.2 Questions About Deferred

Discussion in 'XenForo Questions and Support' started by digitalpoint, Aug 7, 2013.

  1. digitalpoint

    digitalpoint Well-Known Member

    Before 1.2, I disabled cron.php from being triggered by browser activity, and instead set up a crontab job on one of my servers to fire it off every 1 minute. That worked fine... but with 1.2, I decided to go back to how it works by default.

    Is It Atomic?

    After checking some of my web logs, I'm wondering if that was a good idea or not... deferred.php is being called 5-10 times per second (even Googlebot is triggering it to happen)...

    For example, it was kicked off 28 times in this 4 second window (that's not unique, it's like that always).

    Should it really be called that often?
    Code:
    66.249.74.17 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/internet-explorer-8-better-than-firefox.1929807/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    176.35.183.18 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/domain-name-expired-how-long-until-available.10918/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.59.8 (KHTML, like Gecko) Version/5.1.9 Safari/534.59.8"
    122.161.247.237 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/ecommerce.115/" "Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20100101 Firefox/21.0"
    66.249.74.27 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-get-rank-a-web-site-on-search-engine.1893435/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    108.199.254.160 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    66.249.74.45 - - [07/Aug/2013:00:46:05 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/howdy-howdy-hi.415148/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.6 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/offering-work-for-blog-posters.144610/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.45 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/obama-approval-rating-falling.1244046/page-79" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.27 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/making-money-help.1968482/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    137.222.52.4 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/havnt-havent.1152572/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    66.249.74.17 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-ruin-a-web-site.860200/page-4" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    80.239.243.18 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/which-site-can-i-exchange-my-paypal-to-webmoney.1108869/" "Opera/9.80 (BlackBerry; Opera Mini/7.1.32723/30.3593; U; en) Presto/2.8.119 Version/11.10"
    66.249.74.6 - - [07/Aug/2013:00:46:06 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/need-unique-myspace-layouts.128868/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    108.199.254.160 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/le-software-development.1364774/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/need-posters-on-webmasters-forum.1321587/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.28 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/new-free-submission-directory-no-recip.872235/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.17 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/paypal-cheque-not-received-yet-what-should-i-do-suggestion-please.2366498/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    81.140.26.165 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://www.digitalpoint.com/" "Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0"
    60.225.40.112 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/what-does-x64-and-x86-mean.674631/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    66.249.74.45 - - [07/Aug/2013:00:46:07 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/human-traffic-includes-detailed-statistics-country-targetting-0-30-1000-visits.2499188/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    108.199.254.160 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/forums/general-chat.19/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    113.179.29.199 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/threads/best-language-to-start-for-web-development.2637702/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0"
    66.249.74.17 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/increasing-your-adsensing-earnings-200.1289098/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.6 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/huge-traffic-mobile-website-for-sale.1440274/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    193.239.73.222 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 22 "https://forums.digitalpoint.com/conversations/add?to=espmartin" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    66.249.74.7 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/is-this-a-website-or-blog.2004821/" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    66.249.74.28 - - [07/Aug/2013:00:46:08 -0700] "POST /deferred.php HTTP/1.1" 200 33 "https://forums.digitalpoint.com/threads/how-to-recover-penalized-websites.2575169/page-2" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    
    Weirdness On Sub-domains

    I also noticed a route of "deferred" (without the .php) is being called, but only if the user is on a sub-domain. That's not a valid route even on the primary domain, and it's a GET (not a POST). So it's not just an issue of it not using the HTML base properly.

    For example:

    Code:
    66.208.237.49 - - [07/Aug/2013:12:20:16 -0700] "GET /deferred HTTP/1.1" 404 6631 "https://tools.digitalpoint.com/suggestion" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    69.180.14.74 - - [07/Aug/2013:12:20:49 -0700] "GET /deferred HTTP/1.1" 404 6630 "https://tools.digitalpoint.com/tracker" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36"
    66.249.74.218 - - [07/Aug/2013:12:26:37 -0700] "GET /deferred HTTP/1.1" 404 6626 "https://tools.digitalpoint.com/cookie-search?domain=stumpchuck.com" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    I've tried to find where that's coming from, but have been unable to find it (it never requests that URL when on the main sub-domain). Any ideas?
     
    Adam Howard likes this.
  2. Mike

    Mike XenForo Developer Staff Member

    It is atomic (same as cron.php, which probably would've run into the same issues). You can probably keep the cron.php-style system you had, just make sure that you let the deferreds run for longer than 10 seconds at a time (config.php option).

    I'm really not sure about the deferred route. You have all manner of changes in place, so I'm not sure what's going on there.
     
    Claudio likes this.
  3. digitalpoint

    digitalpoint Well-Known Member

    So there is nothing that is deferred that is expected to run instantly with it? I'll set up deferred.php to run once a minute via crontab and see if I break anything.

    Will go digging on the sub-domain thing. Definitely nothing I added, but determined to figure that one out.
     
    Adam Howard likes this.
  4. Mouth

    Mouth Well-Known Member

    What methodology did you use to achieve both of these?
     
  5. digitalpoint

    digitalpoint Well-Known Member

    I suppose it would be useful to post quick resource "how-to" for those that want to. :)
     
    Mouth likes this.
  6. Mouth

    Mouth Well-Known Member

    Much appreciated :)
     
  7. digitalpoint

    digitalpoint Well-Known Member

Share This Page