• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Elasticsearch logfile handling?

HWS

Well-known member
#1
We found hundreds of logfiles in our /elasticsearch/logs directory.
It seems that ES rotates its logs daily, but never prune any of them.

Does anyone know what we have to write into logging.yml to hold only a few days of logs and delete the older ones?
 

Slavik

XenForo moderator
Staff member
#4
Yes, I know. but which setting provides that?
I read that file already before asking here and couldn't find anything.

Can you copy paste the logging.yml file here please, don't have the relevent SSH details with me atm
 

HWS

Well-known member
#5
Code:
[root@web ~]# less /elasticsearch/config/logging.yml
rootLogger: INFO, console, file
logger:
  # log action execution errors for easier debugging
  action: WARN
  # reduce the logging for aws, too much is logged under the default INFO
  com.amazonaws: ERROR
 
  # gateway
  #gateway: DEBUG
  #index.gateway: DEBUG
 
  # peer shard recovery
  #indices.recovery: DEBUG
 
  # discovery
  #discovery: TRACE
 
  index.search.slowlog: TRACE, index_search_slow_log_file
  index.indexing.slowlog: TRACE, index_indexing_slow_log_file
 
additivity:
  index.search.slowlog: false
  index.indexing.slowlog: false
 
appender:
  console:
    type: console
    layout:
      type: consolePattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
 
  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
 
  index_search_slow_log_file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}_index_search_slowlog.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
 
  index_indexing_slow_log_file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}_index_indexing_slowlog.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"

This is the logging.yml.
I can't find anything regarding max old logfiles..
 

Slavik

XenForo moderator
Staff member
#6
This is the logging.yml.
I can't find anything regarding max old logfiles..
Ah yes, now I remember.

The logging file uses log4j format, with the log4j prefixes stripped out.

So the part you would want to change would be

Code:
  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
to something like

Code:
  file:
    type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    maxBackupIndex: 7
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
which would then keep a log of the last 7 days