XF 2.2 Elastic Search Error?

penturners

Active member
A few days ago I started seeing these errors at random times while various actions are being performed. The forum seems to hang for 20 sec for the particular user, then the action seems to complete properly. I can't establish any pattern. Happens when users are posting new or replying to threads, sending conversations, and it happened to me when processing the approval queue.

This one is a user posting a thread. Troubleshooting tips appreciated. Running 2.2.3 patch 1 everything is current.

Code:
     XFES\Elasticsearch\ConnectException: Elasticsearch indexing error (queued): cURL error 28: Operation timed out after 20001 milliseconds with 0 bytes received (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) src/addons/XFES/Elasticsearch/Api.php:391

Code:
#0 src/addons/XFES/Elasticsearch/Api.php(331): XFES\Elasticsearch\Api->request('put', 'dbnameremoved/xf...', '{
    "title": ...')
#1 src/addons/XFES/Elasticsearch/Api.php(349): XFES\Elasticsearch\Api->requestFromIndex('put', 'xf/thread-16886...', Array)
#2 src/addons/XFES/Elasticsearch/Api.php(152): XFES\Elasticsearch\Api->requestById('put', 'thread', 168868, Array)
#3 src/addons/XFES/Search/Source/Elasticsearch.php(54): XFES\Elasticsearch\Api->index('thread', 168868, Array)
#4 src/XF/Search/Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#5 src/XF/Behavior/Indexable.php(46): XF\Search\Search->index('post', Object(XFMG\XF\Entity\Post), false)
#6 src/XF.php(341): XF\Behavior\Indexable->XF\Behavior\{closure}()
#7 src/XF/Mvc/Dispatcher.php(156): XF::triggerRunOnce(true)
#8 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#9 src/XF/App.php(2326): XF\Mvc\Dispatcher->run()
#10 src/XF.php(488): XF\App->run()
#11 index.php(20): XF::runApp('XF\\Pub\\App')
#12 {main}

-------------

Previous GuzzleHttp\Exception\ConnectException: cURL error 28: Operation timed out after 20000 milliseconds with 0 bytes received (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) - src/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php:185
#0 src/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php(149): GuzzleHttp\Handler\CurlFactory::createRejection(Object(GuzzleHttp\Handler\EasyHandle), Array)
#1 src/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php(102): GuzzleHttp\Handler\CurlFactory::finishError(Object(GuzzleHttp\Handler\CurlHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory))
#2 src/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php(43): GuzzleHttp\Handler\CurlFactory::finish(Object(GuzzleHttp\Handler\CurlHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory))
#3 src/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php(28): GuzzleHttp\Handler\CurlHandler->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#4 src/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php(51): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#5 src/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php(66): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#6 src/vendor/guzzlehttp/guzzle/src/Middleware.php(30): GuzzleHttp\PrepareBodyMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#7 src/vendor/guzzlehttp/guzzle/src/RedirectMiddleware.php(70): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#8 src/vendor/guzzlehttp/guzzle/src/Middleware.php(58): GuzzleHttp\RedirectMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#9 src/vendor/guzzlehttp/guzzle/src/HandlerStack.php(67): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array)
#10 src/vendor/guzzlehttp/guzzle/src/Client.php(277): GuzzleHttp\HandlerStack->__invoke(Object(GuzzleHttp\Psr7\Request), Array)
#11 src/vendor/guzzlehttp/guzzle/src/Client.php(98): GuzzleHttp\Client->transfer(Object(GuzzleHttp\Psr7\Request), Array)
#12 src/vendor/guzzlehttp/guzzle/src/Client.php(106): GuzzleHttp\Client->sendAsync(Object(GuzzleHttp\Psr7\Request), Array)
#13 src/addons/XFES/Elasticsearch/Api.php(374): GuzzleHttp\Client->send(Object(GuzzleHttp\Psr7\Request))
#14 src/addons/XFES/Elasticsearch/Api.php(331): XFES\Elasticsearch\Api->request('put', 'penturne_xf2/xf...', '{
    "title": ...')
#15 src/addons/XFES/Elasticsearch/Api.php(349): XFES\Elasticsearch\Api->requestFromIndex('put', 'xf/thread-16886...', Array)
#16 src/addons/XFES/Elasticsearch/Api.php(152): XFES\Elasticsearch\Api->requestById('put', 'thread', 168868, Array)
#17 src/addons/XFES/Search/Source/Elasticsearch.php(54): XFES\Elasticsearch\Api->index('thread', 168868, Array)
#18 src/XF/Search/Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#19 src/XF/Behavior/Indexable.php(46): XF\Search\Search->index('post', Object(XFMG\XF\Entity\Post), false)
#20 src/XF.php(341): XF\Behavior\Indexable->XF\Behavior\{closure}()
#21 src/XF/Mvc/Dispatcher.php(156): XF::triggerRunOnce(true)
#22 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#23 src/XF/App.php(2326): XF\Mvc\Dispatcher->run()
#24 src/XF.php(488): XF\App->run()
#25 index.php(20): XF::runApp('XF\\Pub\\App')
#26 {main}
 
No, this is essentially network issues between the web server and the elasticsearch server.

You need to make sure elasticsearch is running and ensure there is network connectivity between them. Check for firewalls blocking and other network issues, such as the hostname/IP and port number being correct, these are server configuration/network issues. If it’s happening often.
 
It never happened prior to a few days ago. Now it's happening every 15 min or so. Running on a VPS @ Knownhost.

We have a busy forum, so I'm assuming that it's working sometimes.
 
It's this.
 
Last edited:
Interesting that it’s curl28 now instead of curl7, although they both are essentially network issues between the web server and the elasticsearch server. So I guess it’s possible.
 
Problem seems to be solved for me. Apparently the problem started when I did the last XF upgrade, to 2.2.3 p1. Was also running Elasticsearch 6.8.14. Maybe XF 2.2.3 and ES 6.8.14 don't play together, but upgrading to ES 7.11 and configuring the heap size did the trick. Error free for 10 hours now. Thanks for the ideas.
 
I was also experiencing this "cURL 28" error several dozen times a day on my otherwise healthy ElasticSearch instance after migrating to a new server.

Code:
XFES\Elasticsearch\ConnectException: cURL error 28: Resolving timed out after 3511 milliseconds (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) src/addons/XFES/Elasticsearch/Api.php:391

None of the suggestions above fixed it, nor was there anything useful in the ElasticSearch logs. Fun fact: I actually upgraded from ElasticSearch 6 to 7 just to see if it would solve this issue — it didn't. All of this started to point me in the direction of a problem not with ES, but with the server itself.

From my understanding of cURL's Error 28, it's more of a failure to resolve an address before timing out instead of a connection that takes too long to complete. On a whim, I decided to change the address of my ElasticSearch server in the XenForo admin control panel from localhost to 127.0.0.1 and... 24 hours later I have not had any further errors.

My next stop was my server's (CentOS 7) /etc/hosts file, which looked like this:

Code:
127.0.0.1     localhost.localdomain
1.2.3.4       myserverhostname.mydomain.com

This is the default and has not been changed since the server was set up a few months ago. Someone please correct me if I'm wrong, but localhost.localdomain is not the same as localhost and the singular term should be added for completeness.

I can't explain is why just localhost still worked most of the time, but I'm not an expert in how DNS resolutions and the /etc/hosts file works.
 
Top Bottom