ES 2.2 ConnectException cURL 7 - failed to connect to localhost port 9200

[zackw]

Current versions are 2.2.2 for the plugin and 7.10.2 for Elasticsearch itself.
Normally everything runs fine with no logs at all. But starting about Feb 14 to March 9 (today) it's had over 70,200 log entries.
Most all of them are like this:

XFES\Elasticsearch\ConnectException: cURL error 7: Failed to connect to localhost port 9200 after 0 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) src/addons/XFES/Elasticsearch/Api.php:405

Every so often it might be this one:

XFES\Elasticsearch\ConnectException: Elasticsearch indexing error (queued): cURL error 7: Failed to connect to localhost port 9200 after 0 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) src/addons/XFES/Elasticsearch/Api.php:405

And:

XFES\Elasticsearch\ConnectException: Similar thread cache rebuild failure: cURL error 7: Failed to connect to localhost port 9200 after 0 ms: Couldn't connect to server (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) src/addons/XFES/Elasticsearch/Api.php:405

Whatever it is, it seems to be a cURL connect issue. But when I go in SSH I can use cURL with commands like this:

curl http://localhost:9200/_cluster/health?pretty

It works.

I couldn't find where to download the whole log for archiving purposes. I want to clear it out so I can start fresh but I don't want to lose it all either. I can't find where XF is storing the log or what file or where to download. I also looked through settings but couldn't find a way to be emailed/notified when new errors happen. I don't need to log in to the backend very much so sometimes errors get missed for days/weeks because there is no notification. How could I monitor that, especially if XF isn't storing the log in a physical file?

 

[Mouth]

Most all of them are like this

Get the date/time on those, then check your elasticsearch logs for the same/corresponding date/time to see what the issue is.

I couldn't find where to download the whole log for archiving purposes. I want to clear it out so I can start fresh but I don't want to lose it all either. I can't find where XF is storing the log or what file or where to download. I also looked through settings but couldn't find a way to be emailed/notified when new errors happen. I don't need to log in to the backend very much so sometimes errors get missed for days/weeks because there is no notification. How could I monitor that, especially if XF isn't storing the log in a physical file?

You'd need to create a seperate suggestion for those functionalities.

 

[chris p]

Zack, Did you ever figure out what caused these errors. I have all of them shown. I assume that this is a system error of some sort, but it's not obvious what it would be as we have no outgoing block on our firewall. Thanks

 

[zackw]

Zack, Did you ever figure out what caused these errors. I have all of them shown. I assume that this is a system error of some sort, but it's not obvious what it would be as we have no outgoing block on our firewall. Thanks

Well for me a couple times this happened, the service was indeed shut down.

From the command line you can check if java is listening on the port:
netstat -tulnp | grep 9200

You can manage the service with these commands:
service elasticsearch start
service elasticsearch stop
service elasticsearch status

Use status to check, and start it if it's not.

Check logs under /var/log/elasticsearch.

Type typing:
curl http://localhost:9200/_cluster/health?pretty
And
curl -XGET 'http://localhost:9200'

See: https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html#cluster-health

For me, even though the service is in the init.d folder, it doesn't like to autostart after something like a server reboot or a service crash.
Early on I would restart the whole server trying to get everything to reset but it would still be off. I had to start it manually.

Naturally, port 9200 (or whatever) needs to be open in the firewall. /etc/apf/conf.apf.

 

[chris p]

Well for me a couple times this happened, the service was indeed shut down.

From the command line you can check if java is listening on the port:
netstat -tulnp | grep 9200

You can manage the service with these commands:
...

Thanks very much Zack. That does explain it. I applied updates on this Ubuntu system and it probably didn't restart. I'll go check it out and thanks again.