• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

XF 1.4 Changed the deferred.php stuff and now hundreds of server errors

Stuart Wright

Well-known member
#1
I had this in our cron:
* * * * * /home/sites/avforums/public_html/deferred.php > /dev/null 2>&1
then installed this addon:
https://xenforo.com/community/resources/goodfornothing-shell-defer.3791/
And changed the cron accordingly and started getting server errors like this one below.
So I reverted back and uninstalled the addon but now I'm still getting the errors.
About 600 errors every half an hour. E.g.
ErrorException: Undefined property: stdClass::$_type - library/XenES/Search/SourceHandler/ElasticSearch.php:870
Generated By: Unknown Account, 10 minutes ago
Stack Trace
#0 /home/sites/avforums/public_html/library/XenES/Search/SourceHandler/ElasticSearch.php(870): XenForo_Application::handlePhpError(8, 'Undefined prope...', '/home/sites/avf...', 870, Array)
#1 /home/sites/avforums/public_html/library/XenES/Model/Elasticsearch.php(264): XenES_Search_SourceHandler_ElasticSearch->isIndexSuccessful(Object(stdClass))
#2 /home/sites/avforums/public_html/library/XenES/Model/Elasticsearch.php(315): XenES_Model_Elasticsearch->reattemptFailedIndex('index', 'post', 21644366, Array, 5)
#3 /home/sites/avforums/public_html/library/XenES/CronEntry/EsReindex.php(19): XenES_Model_Elasticsearch->runFailedReindexer()
#4 [internal function]: XenES_CronEntry_EsReindex::reindex(Array)
#5 /home/sites/avforums/public_html/library/XenForo/Model/Cron.php(356): call_user_func(Array, Array)
#6 /home/sites/avforums/public_html/library/XenForo/Deferred/Cron.php(24): XenForo_Model_Cron->runEntry(Array)
#7 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(294): XenForo_Deferred_Cron->execute(Array, Array, 7.9999969005585, '')
#8 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(428): XenForo_Model_Deferred->runDeferred(Array, 7.9999969005585, '', false)
#9 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(373): XenForo_Model_Deferred->_runInternal(Array, NULL, '', false)
#10 /home/sites/avforums/public_html/deferred.php(23): XenForo_Model_Deferred->run(false)
#11 {main}
Request State
array(3) {
["url"] => string(37) "https://www.avforums.com/deferred.php"
["_GET"] => array(0) {
}
["_POST"] => array(4) {
["_xfRequestUri"] => string(56) "/review/panasonic-pt-at6000e-3d-lcd-projector-review.384"
["_xfNoRedirect"] => string(1) "1"
["_xfToken"] => string(8) "********"
["_xfResponseType"] => string(4) "json"
}
}
Does anyone know what on earth is going on?
Thanks
 
Last edited:

Daniel Hood

Well-known member
#4
But they're all connected to ElasticSearch? Interesting.. Might need to restart the ES server (I'm not an expert on it though, so don't do it till someone else confirms). I think XenForo Devs may want to consider adding some Exception catching here.
 

Xon

Well-known member
#6
@Stuart Wright Since you ran it as root, make sure you reset file permissions so they are owned by the correct user the website runs as.

The deferred task itself may be screwed up, had have bad data persisted to the xf_deferred table. This may require purging from the DB. You can then restart off the failed indexing per normal.

But looking at that XF Elastic Search code, it is bizarre that it failed there. That is parsing the results from Elastic Search.

Can you check your elastic search instance's logs?
 

HWS

Well-known member
#8
You need to chown all files and folders in /data and /internal_data to your web server user if you ran deferred.php as root to prevent future errors.

Regarding the Elastic Search errors, those are very strange. Never had a similar experience. Maybe you updated Elasticsearch recently and this is not connected with the deferred.php change?
 

Mike

XenForo developer
Staff member
#9
There are actually some changes relating to this ES code but IIRC, it is mostly hiding another error so it wouldn't happen in normal execution unless there was another error happening. I would look at the status of your Elasticsearch server to make sure everything is ok there.
 

Stuart Wright

Well-known member
#11
I've been getting the same errors from Elastic again since Monday.
They all started with this different error:
XenForo_Exception: Elasticsearch indexing failed Elasticsearch error: MapperParsingException[failed to parse, document is empty] - library/XenES/Search/SourceHandler/ElasticSearch.php:840
So it seems that this error triggers an issue which breaks Elastic which then gives all the subsequent errors (4,500+ of them).
Any ideas please?
 

Xon

Well-known member
#12
It sounds like XF is trying to submit an empty post or similar to Elastic Search.

Do you have a stacktrace for that error? It should help suggest what content type is triggering this.
 

Stuart Wright

Well-known member
#13
It sounds like XF is trying to submit an empty post or similar to Elastic Search.

Do you have a stacktrace for that error? It should help suggest what content type is triggering this.
Error Info
XenForo_Exception: Elasticsearch indexing failed Elasticsearch error: MapperParsingException[failed to parse, document is empty] - library/XenES/Search/SourceHandler/ElasticSearch.php:840
Generated By: technomen, Monday at 9:24 AM
Stack Trace
#0 /home/sites/avforums/public_html/library/XenES/Search/SourceHandler/ElasticSearch.php(958): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError('MapperParsingEx...', false, 'Elasticsearch i...')
#1 /home/sites/avforums/public_html/library/XenES/Search/SourceHandler/ElasticSearch.php(63): XenES_Search_SourceHandler_ElasticSearch->_triggerFailedIndexError(Object(stdClass), Array, true)
#2 /home/sites/avforums/public_html/library/XenForo/Search/Indexer.php(44): XenES_Search_SourceHandler_ElasticSearch->insertIntoIndex('post', 21671744, '', 'The real pictur...', 1422264264, 702463, 1931834, Array)
#3 /home/sites/avforums/public_html/library/XenForo/Search/DataHandler/Post.php(61): XenForo_Search_Indexer->insertIntoIndex('post', 21671744, '', 'The real pictur...', 1422264264, 702463, 1931834, Array)
#4 /home/sites/avforums/public_html/library/XenForo/Search/DataHandler/Abstract.php(275): XenForo_Search_DataHandler_Post->_insertIntoIndex(Object(XenForo_Search_Indexer), Array, Array)
#5 /home/sites/avforums/public_html/library/XenForo/DataWriter/DiscussionMessage.php(760): XenForo_Search_DataHandler_Abstract->insertIntoIndex(Object(XenForo_Search_Indexer), Array, Array)
#6 /home/sites/avforums/public_html/library/XenForo/DataWriter/DiscussionMessage.php(739): XenForo_DataWriter_DiscussionMessage->_insertOrUpdateSearchIndex()
#7 /home/sites/avforums/public_html/library/XenForo/DataWriter/DiscussionMessage.php(585): XenForo_DataWriter_DiscussionMessage->_indexForSearch()
#8 /home/sites/avforums/public_html/library/XenForo/DataWriter.php(1409): XenForo_DataWriter_DiscussionMessage->_postSave()
#9 /home/sites/avforums/public_html/library/DigitalPointSpy/DataWriter/DiscussionMessage/Post.php(7): XenForo_DataWriter->save()
#10 /home/sites/avforums/public_html/library/XenForo/ControllerPublic/Thread.php(557): DigitalPointSpy_DataWriter_DiscussionMessage_Post->save()
#11 /home/sites/avforums/public_html/library/XenForo/FrontController.php(347): XenForo_ControllerPublic_Thread->actionAddReply()
#12 /home/sites/avforums/public_html/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#13 /home/sites/avforums/public_html/index.php(13): XenForo_FrontController->run()
#14 {main}
Request State
array(3) {
["url"] => string(87) "https://www.avforums.com/threads/hisense-announce-uled-2-0-4k-uhd-tvs.1931834/add-reply"
["_GET"] => array(1) {
["/threads/hisense-announce-uled-2-0-4k-uhd-tvs_1931834/add-reply"] => string(0) ""
}
["_POST"] => array(9) {
["message"] => string(192) "The real picture is one :
http://www.pcpop.com/doc/1/1067/1067459.shtml
http://www.microsofttranslator.com/...http://www.pcpop.com/doc/1/1067/1067459.shtml
"
["_xfRelativeResolver"] => string(78) "https://www.avforums.com/threads/hisense-announce-uled-2-0-4k-uhd-tvs.1931834/"
["attachment_hash"] => string(32) "4b52a2de5a743b95528bd37ca66db989"
["last_date"] => string(10) "1422215548"
["last_known_date"] => string(10) "1422215548"
["_xfToken"] => string(8) "********"
["_xfRequestUri"] => string(54) "/threads/hisense-announce-uled-2-0-4k-uhd-tvs.1931834/"
["_xfNoRedirect"] => string(1) "1"
["_xfResponseType"] => string(4) "json"
}
}
 

Mike

XenForo developer
Staff member
#14
It looks like a post with invalid UTF-8 in it. It seems that Elasticsearch fell over when it received that. I don't see any obvious bad UTF-8 in the message, but it might be caused by an add-on to convert links to page titles (and it not giving valid UTF-8).