Elasticsearch logfile handling?

[HWS]

We found hundreds of logfiles in our /elasticsearch/logs directory.
It seems that ES rotates its logs daily, but never prune any of them.

Does anyone know what we have to write into logging.yml to hold only a few days of logs and delete the older ones?

 

[Slavik]

edit the config/logging.yml file

 

[HWS]

edit the config/logging.yml file

Yes, I know. but which setting provides that?
I read that file already before asking here and couldn't find anything.

 

[Slavik]

Yes, I know. but which setting provides that?
I read that file already before asking here and couldn't find anything.

Can you copy paste the logging.yml file here please, don't have the relevent SSH details with me atm

 

[HWS]

[root@web ~]# less /elasticsearch/config/logging.yml
rootLogger: INFO, console, file
logger:
  # log action execution errors for easier debugging
  action: WARN
  # reduce the logging for aws, too much is logged under the default INFO
  com.amazonaws: ERROR
 
  # gateway
  #gateway: DEBUG
  #index.gateway: DEBUG
 
  # peer shard recovery
  #indices.recovery: DEBUG
 
  # discovery
  #discovery: TRACE
 
  index.search.slowlog: TRACE, index_search_slow_log_file
  index.indexing.slowlog: TRACE, index_indexing_slow_log_file
 
additivity:
  index.search.slowlog: false
  index.indexing.slowlog: false
 
appender:
  console:
    type: console
    layout:
      type: consolePattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
 
  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
 
  index_search_slow_log_file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}_index_search_slowlog.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
 
  index_indexing_slow_log_file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}_index_indexing_slowlog.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

This is the logging.yml.
I can't find anything regarding max old logfiles..

 

[Slavik]

This is the logging.yml.
I can't find anything regarding max old logfiles..

Ah yes, now I remember.

The logging file uses log4j format, with the log4j prefixes stripped out.

So the part you would want to change would be

  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

to something like

  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    maxBackupIndex: 7
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

which would then keep a log of the last 7 days

 

[HWS]

which would then keep a log of the last 7 days

Thank you!