XF 1.4 Changed the deferred.php stuff and now hundreds of server errors

Stuart Wright

Stuart Wright

Well-known member
I had this in our cron:
* * * * * /home/sites/avforums/public_html/deferred.php > /dev/null 2>&1
then installed this addon:
https://xenforo.com/community/resources/goodfornothing-shell-defer.3791/
And changed the cron accordingly and started getting server errors like this one below.
So I reverted back and uninstalled the addon but now I'm still getting the errors.
About 600 errors every half an hour. E.g.
ErrorException: Undefined property: stdClass::$_type - library/XenES/Search/SourceHandler/ElasticSearch.php:870
Generated By: Unknown Account, 10 minutes ago
Stack Trace
#0 /home/sites/avforums/public_html/library/XenES/Search/SourceHandler/ElasticSearch.php(870): XenForo_Application::handlePhpError(8, 'Undefined prope...', '/home/sites/avf...', 870, Array)
#1 /home/sites/avforums/public_html/library/XenES/Model/Elasticsearch.php(264): XenES_Search_SourceHandler_ElasticSearch->isIndexSuccessful(Object(stdClass))
#2 /home/sites/avforums/public_html/library/XenES/Model/Elasticsearch.php(315): XenES_Model_Elasticsearch->reattemptFailedIndex('index', 'post', 21644366, Array, 5)
#3 /home/sites/avforums/public_html/library/XenES/CronEntry/EsReindex.php(19): XenES_Model_Elasticsearch->runFailedReindexer()
#4 [internal function]: XenES_CronEntry_EsReindex::reindex(Array)
#5 /home/sites/avforums/public_html/library/XenForo/Model/Cron.php(356): call_user_func(Array, Array)
#6 /home/sites/avforums/public_html/library/XenForo/Deferred/Cron.php(24): XenForo_Model_Cron->runEntry(Array)
#7 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(294): XenForo_Deferred_Cron->execute(Array, Array, 7.9999969005585, '')
#8 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(428): XenForo_Model_Deferred->runDeferred(Array, 7.9999969005585, '', false)
#9 /home/sites/avforums/public_html/library/XenForo/Model/Deferred.php(373): XenForo_Model_Deferred->_runInternal(Array, NULL, '', false)
#10 /home/sites/avforums/public_html/deferred.php(23): XenForo_Model_Deferred->run(false)
#11 {main}
Request State
array(3) {
["url"] => string(37) "https://www.avforums.com/deferred.php"
["_GET"] => array(0) {
}
["_POST"] => array(4) {
["_xfRequestUri"] => string(56) "/review/panasonic-pt-at6000e-3d-lcd-projector-review.384"
["_xfNoRedirect"] => string(1) "1"
["_xfToken"] => string(8) "********"
["_xfResponseType"] => string(4) "json"
}
}
Click to expand...
Does anyone know what on earth is going on?
Thanks
 
Last edited:
Sort by date Sort by votes
But they're all connected to ElasticSearch? Interesting.. Might need to restart the ES server (I'm not an expert on it though, so don't do it till someone else confirms). I think XenForo Devs may want to consider adding some Exception catching here.
 
Upvote 0 Downvote
@Stuart Wright Since you ran it as root, make sure you reset file permissions so they are owned by the correct user the website runs as.

The deferred task itself may be screwed up, had have bad data persisted to the xf_deferred table. This may require purging from the DB. You can then restart off the failed indexing per normal.

But looking at that XF Elastic Search code, it is bizarre that it failed there. That is parsing the results from Elastic Search.

Can you check your elastic search instance's logs?
 
Upvote 0 Downvote
You need to chown all files and folders in /data and /internal_data to your web server user if you ran deferred.php as root to prevent future errors.

Regarding the Elastic Search errors, those are very strange. Never had a similar experience. Maybe you updated Elasticsearch recently and this is not connected with the deferred.php change?
 
Upvote 0 Downvote
There are actually some changes relating to this ES code but IIRC, it is mostly hiding another error so it wouldn't happen in normal execution unless there was another error happening. I would look at the status of your Elasticsearch server to make sure everything is ok there.
 
  • Like
Reactions: Xon
Upvote 0 Downvote
I've been getting the same errors from Elastic again since Monday.
They all started with this different error:
XenForo_Exception: Elasticsearch indexing failed Elasticsearch error: MapperParsingException[failed to parse, document is empty] - library/XenES/Search/SourceHandler/ElasticSearch.php:840
Click to expand...
So it seems that this error triggers an issue which breaks Elastic which then gives all the subsequent errors (4,500+ of them).
Any ideas please?
 
Upvote 0 Downvote
It sounds like XF is trying to submit an empty post or similar to Elastic Search.

Do you have a stacktrace for that error? It should help suggest what content type is triggering this.
 
Upvote 0 Downvote
Xon said:
It sounds like XF is trying to submit an empty post or similar to Elastic Search.

Do you have a stacktrace for that error? It should help suggest what content type is triggering this.
Click to expand...
Error Info
XenForo_Exception: Elasticsearch indexing failed Elasticsearch error: MapperParsingException[failed to parse, document is empty] - library/XenES/Search/SourceHandler/ElasticSearch.php:840
Generated By: technomen, Monday at 9:24 AM
Stack Trace
#0 /home/sites/avforums/public_html/library/XenES/Search/SourceHandler/ElasticSearch.php(958): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError('MapperParsingEx...', false, 'Elasticsearch i...')
#1 /home/sites/avforums/public_html/library/XenES/Search/SourceHandler/ElasticSearch.php(63): XenES_Search_SourceHandler_ElasticSearch->_triggerFailedIndexError(Object(stdClass), Array, true)
#2 /home/sites/avforums/public_html/library/XenForo/Search/Indexer.php(44): XenES_Search_SourceHandler_ElasticSearch->insertIntoIndex('post', 21671744, '', 'The real pictur...', 1422264264, 702463, 1931834, Array)
#3 /home/sites/avforums/public_html/library/XenForo/Search/DataHandler/Post.php(61): XenForo_Search_Indexer->insertIntoIndex('post', 21671744, '', 'The real pictur...', 1422264264, 702463, 1931834, Array)
#4 /home/sites/avforums/public_html/library/XenForo/Search/DataHandler/Abstract.php(275): XenForo_Search_DataHandler_Post->_insertIntoIndex(Object(XenForo_Search_Indexer), Array, Array)
#5 /home/sites/avforums/public_html/library/XenForo/DataWriter/DiscussionMessage.php(760): XenForo_Search_DataHandler_Abstract->insertIntoIndex(Object(XenForo_Search_Indexer), Array, Array)
#6 /home/sites/avforums/public_html/library/XenForo/DataWriter/DiscussionMessage.php(739): XenForo_DataWriter_DiscussionMessage->_insertOrUpdateSearchIndex()
#7 /home/sites/avforums/public_html/library/XenForo/DataWriter/DiscussionMessage.php(585): XenForo_DataWriter_DiscussionMessage->_indexForSearch()
#8 /home/sites/avforums/public_html/library/XenForo/DataWriter.php(1409): XenForo_DataWriter_DiscussionMessage->_postSave()
#9 /home/sites/avforums/public_html/library/DigitalPointSpy/DataWriter/DiscussionMessage/Post.php(7): XenForo_DataWriter->save()
#10 /home/sites/avforums/public_html/library/XenForo/ControllerPublic/Thread.php(557): DigitalPointSpy_DataWriter_DiscussionMessage_Post->save()
#11 /home/sites/avforums/public_html/library/XenForo/FrontController.php(347): XenForo_ControllerPublic_Thread->actionAddReply()
#12 /home/sites/avforums/public_html/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#13 /home/sites/avforums/public_html/index.php(13): XenForo_FrontController->run()
#14 {main}
Request State
array(3) {
["url"] => string(87) "https://www.avforums.com/threads/hisense-announce-uled-2-0-4k-uhd-tvs.1931834/add-reply"
["_GET"] => array(1) {
["/threads/hisense-announce-uled-2-0-4k-uhd-tvs_1931834/add-reply"] => string(0) ""
}
["_POST"] => array(9) {
["message"] => string(192) "The real picture is one :
http://www.pcpop.com/doc/1/1067/1067459.shtml
http://www.microsofttranslator.com/...http://www.pcpop.com/doc/1/1067/1067459.shtml
"
["_xfRelativeResolver"] => string(78) "https://www.avforums.com/threads/hisense-announce-uled-2-0-4k-uhd-tvs.1931834/"
["attachment_hash"] => string(32) "4b52a2de5a743b95528bd37ca66db989"
["last_date"] => string(10) "1422215548"
["last_known_date"] => string(10) "1422215548"
["_xfToken"] => string(8) "********"
["_xfRequestUri"] => string(54) "/threads/hisense-announce-uled-2-0-4k-uhd-tvs.1931834/"
["_xfNoRedirect"] => string(1) "1"
["_xfResponseType"] => string(4) "json"
}
}
Click to expand...
 
Upvote 0 Downvote
It looks like a post with invalid UTF-8 in it. It seems that Elasticsearch fell over when it received that. I don't see any obvious bad UTF-8 in the message, but it might be caused by an add-on to convert links to page titles (and it not giving valid UTF-8).
 
  • Like
Reactions: Xon
Upvote 0 Downvote
You must log in or register to reply here.

Similar threads

N
  • Question Question
XF 1.3 Tons of server errors!
Replies
6
Views
1K
NeoCHI
N
Back
Top Bottom