How to: Basic Elasticsearch installation. (Debian/Ubuntu)

Slavik

XenForo moderator
Staff member
Basic Elasticsearch Installation (Debian/Ubuntu)

This guide is provided to show how to do a basic (vanilla get up and go) install of Elasticsearch (0.18.7), the Elasticsearch Service Wrapper and the required Java Runtime Environment (JRE) (1.7.0_02) on Debian/Ubuntu. This guide will not cover running a dedicated Elasticsearch user.

For RHEL/SUSE users, a guide can be found here.

This guide assumes the user has basic knowledge of SSH and prior to starting the steps below has logged in as root.

Please note, whilst this is a simple and easy setup, I take no responsibility for any damages or losses that may occur to your system by following the steps below. If you are unsure at any stage, please ask for assistance or seek the help of a qualified Linux Systems Administrator.

Installing the JRE
Type the following commands into your SSH terminal.

Code:
apt-get install openjdk-7-jre-headless
java -version

Assuming everything was done correctly, you should get a similar output to:

Code:
# java -version
java version "1.7.0_02"
Java(TM) SE Runtime Environment (build 1.7.0_02-b13)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode)

Install Elasticsearch

Code:
cd /
curl -OL -k http://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.18.7.zip
unzip elasticsearch-0.18.7.zip
mv elasticsearch-0.18.7 /usr/local/elasticsearch

Install the Elasticsearch Service Wrapper

Create a new file in /etc/init.d/ named elasticsearch

Open /etc/init.d/elasticsearch and paste the following code in.

Code:
#! /bin/sh
### BEGIN INIT INFO
# Provides:          elasticsearch
# Required-Start:    $all
# Required-Stop:    $all
# Default-Start:    2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Starts elasticsearch
# Description:      Starts elasticsearch using start-stop-daemon
### END INIT INFO
 
ES_HOME=/usr/local/elasticsearch
ES_MIN_MEM=256m
ES_MAX_MEM=1g
DAEMON=$ES_HOME/bin/elasticsearch
NAME=elasticsearch
DESC=elasticsearch
PID_FILE=/var/run/$NAME.pid
LOG_DIR=/var/log/$NAME
DATA_DIR=/var/lib/$NAME
WORK_DIR=/tmp/$NAME
CONFIG_FILE=/etc/$NAME/elasticsearch.yml
DAEMON_OPTS="-p $PID_FILE -Des.config=$CONFIG_FILE -Des.path.home=$ES_HOME -Des.path.logs=$LOG_DIR -Des.path.data=$DATA_DIR -Des.path.work=$WORK_DIR"
 
 
test -x $DAEMON || exit 0
 
set -e
 
case "$1" in
  start)
    echo -n "Starting $DESC: "
    mkdir -p $LOG_DIR $DATA_DIR $WORK_DIR
    if start-stop-daemon --start --pidfile $PID_FILE --startas $DAEMON -- $DAEMON_OPTS
    then
        echo "started."
    else
        echo "failed."
    fi
    ;;
  stop)
    echo -n "Stopping $DESC: "
    if start-stop-daemon --stop --pidfile $PID_FILE
    then
        echo "stopped."
    else
        echo "failed."
    fi
    ;;
  restart|force-reload)
    ${0} stop
    sleep 0.5
    ${0} start
    ;;
  *)
    N=/etc/init.d/$NAME
    echo "Usage: $N {start|stop|restart|force-reload}" >&2
    exit 1
    ;;
esac
 
exit 0

Save and Exit.

Make the file executable.

Code:
chmod +x /etc/init.d/elasticsearch

Install the script

Code:
update-rc.d elasticsearch defaults

Basic Configuration

You should do some basic configuration of Elasticsearch before installing the addon in XenForo.


1) Create a file at /etc/elasticsearch/ named elasticsearch.yml and paste the following in.

Code:
cluster.name: PUT-SOMETHING-UNIQUE-HERE

Save and Close


2) Open up /etc/init.d/elasticsearch on line 13+14 edit

Code:
ES_MIN_MEM=256m
ES_MAX_MEM=1g

To a number suitable for the size of your forum.

I reccomend approximately 256mb for the MIN_MEM and 1 GB for the MAX_MEM per 1 million posts on your forum.

1 Million Posts: 256m:1g
2 Million Posts: 512m:2g
3 Million Posts: 768m:3g
4 Million Posts: 1024m:4g
etc

This will not mean the service will use all that available memory, however it will have it at its disposal if required.

So for example a 3 Million Post forum would edit

Code:
ES_MIN_MEM=256m
 
ES_MAX_MEM=1g

to

Code:
ES_MIN_MEM=768m
 
ES_MAX_MEM=3g

Save and Exit.


3) Optional - Move the Elasticsearch data directory.

Your linux install may be configured in such a way that your install partition is only a few Gb in size, and placing a large Elasticsearch index there is not ideal.

In which case you will want to move the index directory to a different, larger, location (in this example /var/elasticsearch)

Code:
cd /var
mkdir elasticsearch

Open up /usr/local/elasticsearch/config/elasticsearch.yml on line 143 edit

Code:
# path.data: /path/to/data

to

Code:
path.data: /var/elasticsearch


Save and Exit


4) Restart the Elasticsearch Service

In SSH type

Code:
/etc/init.d/elasticsearch restart

You should get the following output

Code:
/etc/init.d/elasticsearch restart
Stopping ElasticSearch...
Stopped ElasticSearch.
Starting ElasticSearch...
Waiting for ElasticSearch......
running: PID: xxxxx

Elasticsearch is now runing with your updated config.

Install the XenForo Enhanced Search Addon

1) Turn your board off into maintainance mode*

2) Download the addon from your customer area at http://xenforo.com/customers/

3) Follow the instructions found at http://xenforo.com/help/enhanced-search/

4) Wait for your indexes to be rebuilt

5) Open your board.

*You may leave your board open during the re-index process, however until items have been indexed they will be unavailable to your users if searched for. Likewise the indexing process is rather server intensive, and as such this is my personal recomendation and not a requirement.


Congratulations. Your board should now be running XenForo Enhanced Search.
 
Now even easier again to install ElasticSearch (and automatically keep up-to-date with latest version) onto your Debian/Ubuntu server ...
  1. sudo wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add -
  2. sudo echo "deb http://packages.elasticsearch.org/elasticsearch/0.90/debian stable main" > /etc/apt/sources.list.d/elastic_search.list
  3. sudo apt-get update
  4. sudo apt-get install elasticsearch
Follow the instructions in first post from the section Basic Configuration onwards
 
You will not have problems with the functionality, but rather with the proper architecture of the package.
Which explains why nobody released any packages for elasticsearch (RHEL, Fedora, Debian etc.). I'm not comfortable with a package forcing to install things on wrong location or to add extra files or libraries not needed.
 
{
"status" : 200,
"name" : "Eddie Brock",
"version" : {
"number" : "1.0.0",
"build_hash" : "a46900e9c72c0a623d71b54016357d5f94c8ea32",
"build_timestamp" : "2014-02-12T16:18:34Z",
"build_snapshot" : false,
"lucene_version" : "4.6"
},
"tagline" : "You Know, for Search"
}

Downloaded elasticsearch-1.0.0 to temp directory, untarred, stopped elasticsearch, overwrote files in (ES_HOME=...) location with the new 1.0.0 files, removed all previous nodes/indexes, restarted, upgraded XFES to 1.0.3 (for all sites), reset elasticsearch settings in ACP, then rebuilt search indexes. Two birds with one stone. (y)
 
The deb package looks useable.

rpm -i elasticsearch-1.1.1.deb

#Done.
#Check periodically whether there is a new version on elasticsearch.org
Important: close port 9200 incomming to your public IP. Otherwise your ElasticSearch REST API is fully open to the web (default!), everbody is able to change your elasticsearch installation/index!
Example: /sbin/iptables -D INPUT -d 99.129.129.99 -p tcp --destination-port 9200 -j DROP

Change ElasticSearch config (raise JVM heap for example):
/etc/default/elasticsearch
Index Location default:
/var/lib/elasticsearch/elasticsearch/nodes/0/indices
Prefered ES view/manage GUI: https://github.com/mobz/elasticsearch-head
 
@Slavik - My web administrator is asking for 'data path' for the elasticsearch. He's performing an upgrade and asked for that information. I'm totally clueless.
/var/lib/elasticsearch/ is the default.
Check either /etc/default/elasticsearch or /etc/elasticsearch/elasticsearch.yml for "data_dir"
Alternatively, "locate node.lock" should find it for you too.
 
Top Bottom