[2.0 RC1] ElasticSearch running but can't reindex search index while Enhanced Search enabled!

[Andro]

Everything was fine until i upgraded to XF 2 RC1 from Beta 8.

If i try to rebuild search index with elasticsearch enabled the following errors comes and reindexing stops:

Errors which getting logged:

Spoiler: Error1

XFES\Elasticsearch\ConnectException: Elasticsearch indexing error: cURL error 28: Operation timed out after 20000 milliseconds with 0 bytes received src/addons/XFES/Elasticsearch/Api.php:368
Generated by: x Nov 8, 2017 at 9:41 PM
Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(397): XFES\Elasticsearch\Api->request('post', '_bulk', '{"index":{"_ind...', Array, Object(GuzzleHttp\Message\Request))
#1 src/addons/XFES/Elasticsearch/Api.php(171): XFES\Elasticsearch\Api->bulkRequest('{"index":{"_ind...')
#2 src/addons/XFES/Search/Source/Elasticsearch.php(82): XFES\Elasticsearch\Api->indexBulk(Array)
#3 src/addons/XFES/Search/Source/Elasticsearch.php(57): XFES\Search\Source\Elasticsearch->flushBulkIndexing()
#4 src/XF/Search/Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#5 src/XF/Search/Search.php(59): XF\Search\Search->index('post', Object(XF\Entity\Post))
#6 src/XF/Search/Search.php(85): XF\Search\Search->indexEntities('post', Object(XF\Mvc\Entity\ArrayCollection))
#7 src/XF/Job/SearchRebuild.php(57): XF\Search\Search->indexRange('post', 0, '500')
#8 src/XF/Job/Manager.php(241): XF\Job\SearchRebuild->run(8)
#9 src/XF/Job/Manager.php(187): XF\Job\Manager->runJobInternal(Array, 8)
#10 src/XF/Job/Manager.php(76): XF\Job\Manager->runJobEntry(Array, 8)
#11 src/XF/Admin/Controller/Tools.php(139): XF\Job\Manager->runQueue(true, 8)
#12 src/XF/Mvc/Dispatcher.php(249): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#13 src/XF/Mvc/Dispatcher.php(88): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', 'html', Object(XF\Mvc\ParameterBag), 'tools', Object(XF\Admin\Controller\Tools), NULL)
#14 src/XF/Mvc/Dispatcher.php(41): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#15 src/XF/App.php(1852): XF\Mvc\Dispatcher->run()
#16 src/XF.php(328): XF\App->run()
#17 admin.php(13): XF::runApp('XF\\Admin\\App')
#18 {main}

-------------

Previous GuzzleHttp\Exception\ConnectException: cURL error 28: Operation timed out after 20000 milliseconds with 0 bytes received - src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:49
#0 src/vendor/guzzlehttp/guzzle/src/RequestFsm.php(103): GuzzleHttp\Exception\RequestException::wrapException(Object(GuzzleHttp\Message\Request), Object(GuzzleHttp\Ring\Exception\ConnectException))
#1 src/vendor/guzzlehttp/guzzle/src/RequestFsm.php(132): GuzzleHttp\RequestFsm->__invoke(Object(GuzzleHttp\Transaction))
#2 src/vendor/react/promise/src/FulfilledPromise.php(25): GuzzleHttp\RequestFsm->GuzzleHttp\{closure}(Array)
#3 src/vendor/guzzlehttp/ringphp/src/Future/CompletedFutureValue.php(55): React\Promise\FulfilledPromise->then(Object(Closure), NULL, NULL)
#4 src/vendor/guzzlehttp/guzzle/src/Message/FutureResponse.php(43): GuzzleHttp\Ring\Future\CompletedFutureValue->then(Object(Closure), NULL, NULL)
#5 src/vendor/guzzlehttp/guzzle/src/RequestFsm.php(134): GuzzleHttp\Message\FutureResponse:roxy(Object(GuzzleHttp\Ring\Future\CompletedFutureArray), Object(Closure))
#6 src/vendor/guzzlehttp/guzzle/src/Client.php(165): GuzzleHttp\RequestFsm->__invoke(Object(GuzzleHttp\Transaction))
#7 src/addons/XFES/Elasticsearch/Api.php(351): GuzzleHttp\Client->send(Object(GuzzleHttp\Message\Request))
#8 src/addons/XFES/Elasticsearch/Api.php(397): XFES\Elasticsearch\Api->request('post', '_bulk', '{"index":{"_ind...', Array, Object(GuzzleHttp\Message\Request))
#9 src/addons/XFES/Elasticsearch/Api.php(171): XFES\Elasticsearch\Api->bulkRequest('{"index":{"_ind...')
#10 src/addons/XFES/Search/Source/Elasticsearch.php(82): XFES\Elasticsearch\Api->indexBulk(Array)
#11 src/addons/XFES/Search/Source/Elasticsearch.php(57): XFES\Search\Source\Elasticsearch->flushBulkIndexing()
#12 src/XF/Search/Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#13 src/XF/Search/Search.php(59): XF\Search\Search->index('post', Object(XF\Entity\Post))
#14 src/XF/Search/Search.php(85): XF\Search\Search->indexEntities('post', Object(XF\Mvc\Entity\ArrayCollection))
#15 src/XF/Job/SearchRebuild.php(57): XF\Search\Search->indexRange('post', 0, '500')
#16 src/XF/Job/Manager.php(241): XF\Job\SearchRebuild->run(8)
#17 src/XF/Job/Manager.php(187): XF\Job\Manager->runJobInternal(Array, 8)
#18 src/XF/Job/Manager.php(76): XF\Job\Manager->runJobEntry(Array, 8)
#19 src/XF/Admin/Controller/Tools.php(139): XF\Job\Manager->runQueue(true, 8)
#20 src/XF/Mvc/Dispatcher.php(249): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#21 src/XF/Mvc/Dispatcher.php(88): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', 'html', Object(XF\Mvc\ParameterBag), 'tools', Object(XF\Admin\Controller\Tools), NULL)
#22 src/XF/Mvc/Dispatcher.php(41): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#23 src/XF/App.php(1852): XF\Mvc\Dispatcher->run()
#24 src/XF.php(328): XF\App->run()
#25 admin.php(13): XF::runApp('XF\\Admin\\App')
#26 {main}

-------------

Previous GuzzleHttp\Ring\Exception\ConnectException: cURL error 28: Operation timed out after 20000 milliseconds with 0 bytes received - src/vendor/guzzlehttp/ringphp/src/Client/CurlFactory.php:126
#0 src/vendor/guzzlehttp/ringphp/src/Client/CurlFactory.php(91): GuzzleHttp\Ring\Client\CurlFactory::createErrorResponse(Array, Array, Array)
#1 src/vendor/guzzlehttp/ringphp/src/Client/CurlHandler.php(96): GuzzleHttp\Ring\Client\CurlFactory::createResponse(Array, Array, Array, Array, Resource id #10)
#2 src/vendor/guzzlehttp/ringphp/src/Client/CurlHandler.php(68): GuzzleHttp\Ring\Client\CurlHandler->_invokeAsArray(Array)
#3 src/vendor/guzzlehttp/ringphp/src/Client/Middleware.php(54): GuzzleHttp\Ring\Client\CurlHandler->__invoke(Array)
#4 src/vendor/guzzlehttp/ringphp/src/Client/Middleware.php(30): GuzzleHttp\Ring\Client\Middleware::GuzzleHttp\Ring\Client\{closure}(Array)
#5 src/vendor/guzzlehttp/guzzle/src/RequestFsm.php(129): GuzzleHttp\Ring\Client\Middleware::GuzzleHttp\Ring\Client\{closure}(Array)
#6 src/vendor/guzzlehttp/guzzle/src/Client.php(165): GuzzleHttp\RequestFsm->__invoke(Object(GuzzleHttp\Transaction))
#7 src/addons/XFES/Elasticsearch/Api.php(351): GuzzleHttp\Client->send(Object(GuzzleHttp\Message\Request))
#8 src/addons/XFES/Elasticsearch/Api.php(397): XFES\Elasticsearch\Api->request('post', '_bulk', '{"index":{"_ind...', Array, Object(GuzzleHttp\Message\Request))
#9 src/addons/XFES/Elasticsearch/Api.php(171): XFES\Elasticsearch\Api->bulkRequest('{"index":{"_ind...')
#10 src/addons/XFES/Search/Source/Elasticsearch.php(82): XFES\Elasticsearch\Api->indexBulk(Array)
#11 src/addons/XFES/Search/Source/Elasticsearch.php(57): XFES\Search\Source\Elasticsearch->flushBulkIndexing()
#12 src/XF/Search/Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#13 src/XF/Search/Search.php(59): XF\Search\Search->index('post', Object(XF\Entity\Post))
#14 src/XF/Search/Search.php(85): XF\Search\Search->indexEntities('post', Object(XF\Mvc\Entity\ArrayCollection))
#15 src/XF/Job/SearchRebuild.php(57): XF\Search\Search->indexRange('post', 0, '500')
#16 src/XF/Job/Manager.php(241): XF\Job\SearchRebuild->run(8)
#17 src/XF/Job/Manager.php(187): XF\Job\Manager->runJobInternal(Array, 8)
#18 src/XF/Job/Manager.php(76): XF\Job\Manager->runJobEntry(Array, 8)
#19 src/XF/Admin/Controller/Tools.php(139): XF\Job\Manager->runQueue(true, 8)
#20 src/XF/Mvc/Dispatcher.php(249): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#21 src/XF/Mvc/Dispatcher.php(88): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', 'html', Object(XF\Mvc\ParameterBag), 'tools', Object(XF\Admin\Controller\Tools), NULL)
#22 src/XF/Mvc/Dispatcher.php(41): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#23 src/XF/App.php(1852): XF\Mvc\Dispatcher->run()
#24 src/XF.php(328): XF\App->run()
#25 admin.php(13): XF::runApp('XF\\Admin\\App')
#26 {main}
Request state
array(4) {
["url"] => string(24) "/admin.php?tools/run-job"
["referrer"] => string(44) "/admin.php?tools/run-job"
["_GET"] => array(1) {
["tools/run-job"] => string(0) ""
}
["_POST"] => array(3) {
["_xfRedirect"] => string(30) "/admin.php"
["_xfToken"] => string(8) "********"
["only_ids"] => string(0) ""
}
}

Spoiler: Error2

XFES\Elasticsearch\RequestException: Elasticsearch error: all shards failed src/addons/XFES/Elasticsearch/Api.php:375
Generated by: Unknown account Nov 8, 2017 at 9:35 PM
Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(312): XFES\Elasticsearch\Api->request('get', '_xenfor...', '{\n "sort": [...')
#1 src/addons/XFES/Elasticsearch/Api.php(216): XFES\Elasticsearch\Api->requestFromIndex('get', '_search', Array)
#2 src/addons/XFES/Search/Source/Elasticsearch.php(149): XFES\Elasticsearch\Api->search(Array)
#3 src/XF/Search/Search.php(164): XFES\Search\Source\Elasticsearch->search(Object(XF\Search\Query\Query), '200')
#4 src/XF/Pub/Controller/Member.php(527): XF\Search\Search->search(Object(XF\Search\Query\Query))
#5 src/XF/Mvc/Dispatcher.php(249): XF\Pub\Controller\Member->actionRecentContent(Object(XF\Mvc\ParameterBag))
#6 src/XF/Mvc/Dispatcher.php(88): XF\Mvc\Dispatcher->dispatchClass('XF:Member', 'RecentContent', 'html', Object(XF\Mvc\ParameterBag), 'members', Object(XF\Pub\Controller\Member), NULL)
#7 src/XF/Mvc/Dispatcher.php(41): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#8 src/XF/App.php(1852): XF\Mvc\Dispatcher->run()
#9 src/XF.php(328): XF\App->run()
#10 index.php(13): XF::runApp('XF\\Pub\\App')
#11 {main}
Request state
array(4) {
["url"] => string(38) "/members/8372/recent-content"
["referrer"] => bool(false)
["_GET"] => array(0) {
}
["_POST"] => array(0) {
}
}

Any kind of help would be appreciated.

 

[Mike]

Nothing changed between beta 8 and RC1. These look like more fundamental errors from your Elasticsearch server -- it didn't respond in 20 seconds, for example. Then it also said "all shards failed", so you would need to look at the Elasticsearch logs to see what it's reporting and will likely need to take action there.