ES 2.2 Elastic monitor and restart with WHM chkservd service, radom stops

[philmckrackon]

My service randomly stops an my java logs are empty so I want to monitor via chkservd until I figure this out.

I have added the Elastic monnittor to
in /etc/chkserv.d/chkservd.conf
as
elasticsearch:1

and also vied (editor)
elasticsearch in /etc/chkserv.d
with line
service[elasticsearch]=9200,curl -XGET 'http://localhost:9200',200,/bin/systemctl restart elasticsearch.service,elasticsearch,root

Am I querying correctly?

If not, what should the query be as well as tthe rretun?

THanks,

 

[philmckrackon]

Thoughts? Is '200' the correct return value?

 

[Travis-Mc]

Did this work for you?

 

[philmckrackon]

It works now that I found a list of return codes here -->> https://curl.se/libcurl/c/libcurl-errors.html and changed the expected return code from 200 to 0

service[elasticsearch]=9200,curl -XGET 'http://localhost:9200',200,/bin/systemctl restart elasticsearch.service,elasticsearch,root

to

service[elasticsearch]=9200,curl -XGET 'http://localhost:9200',0,/bin/systemctl restart elasticsearch.service,elasticsearch,root

This is the ftp example I modified. -->> https://support.cpanel.net/hc/en-us...-to-add-a-new-service-for-chkservd-to-monitor

 

[Travis-Mc]

That is very helpful. Thank you so much.

 

[Travis-Mc]

@philmckrackon I am not able to get this to fully work. I keep getting false positives of the service being down when it is not. Did you happen to run into that issue?

 

[philmckrackon]

What I posted works for me but I am on CentOS 7. What is your OS and what is your curl statement? It may be different and may not need the ' '. I would use a ssh terminal to check all your commands and modify as needed.

 

[Travis-Mc]

CentOS 7 as well. I tried each of these one at a time:

service[elasticsearch]=9200,curl -XGET 'http://localhost:9200',0,/bin/systemctl restart elasticsearch.service,elasticsearch,root
service[elasticsearch]=9200,curl -XGET 'http://127.0.0.1:9200',0,/bin/systemctl restart elasticsearch.service,elasticsearch,root
service[elasticsearch]=x,x,x,/bin/systemctl restart elasticsearch.service,elasticsearch,root

I also restarted chkservd and elasticsearch with each change.

In the file /var/run/chkservd/elasticsearch I changed the - to a +, but that didn't seem to do anything either.

Everything done via SSH

 

[Travis-Mc]

The command

# curl -XGET 'http://localhost:9200'

Returns this (I changed some things to ***** because I don't know what is okay to display publicly):

{
  "name" : "*****",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "*****",
  "version" : {
    "number" : "7.16.2",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "*****",
    "build_date" : "2021-12-18T19:42:46.604893745Z",
    "build_snapshot" : false,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

 

[philmckrackon]

Sorry for the late reply, apparently WHM did an update an now elasticsearch is restarting all the time killings my resources - once I get this tackled I'll t back...

 

[Travis-Mc]

I gave up trying to figure this one out for now. If you come up with something, let me know.