How to: Basic Elasticsearch installation. (RHEL/SUSE)

Slavik

XenForo moderator
Staff member
Basic Elasticsearch Installation (RHEL / SUSE)

@Floren has an amazing repo for RHEL 6 and 7, the Elasticsearch RPM he provides is perfectly set up and currently I suggest using it.

The old manual setup guide can be found below.

Step 1) Install the Axivo Repo: https://www.axivo.com/resources/repository-setup.1/

Step 2) Install ElasticSearch: https://www.axivo.com/resources/elasticsearch-setup.11/




This guide is provided to show how to do a basic (vanilla get up and go) install of Elasticsearch (0.90.0 Beta 1), the Elasticsearch Service Wrapper and the required Java Runtime Environment (JRE) (1.7.0_17) on RHEL / SUSE. This guide will not cover running a dedicated Elasticsearch user.

For Debian/Ubuntu users, a guide can be found here.

This guide assumes the user has basic knowledge of SSH and prior to starting the steps below has logged in as root. This guide also assumes the user does not currently have any JRE installed. You can check if you have JRE installed by typing

Code:
java -version

As of writing, the current file locations for JRE are as follows:

32 bit
Code:
http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-i586.rpm

64 bit
Code:
http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-x64.rpm

The guide will be shown using the 64 bit install, however if you are using a 32 bit system, change the file names as appropriate.

Please note, whilst this is a simple and easy setup, I take no responsibility for any damages or losses that may occur to your system by following the steps below. If you are unsure at any stage, please ask for assistance or seek the help of a qualified Linux Systems Administrator.

Installing the JRE
Type the following commands into your SSH terminal.
Code:
cd /tmp
wget http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-x64.rpm
rpm -ivh jre-7u17-linux-x64.rpm
java -version

Assuming everything was done correctly, you should get the following output.

Code:
# java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode)

Install Elasticsearch

Code:
cd /
curl -L -O -k https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.0.Beta1.zip
unzip elasticsearch-0.90.0.Beta1.zip
mv elasticsearch-0.90.0.Beta1 elasticsearch

Install the Elasticsearch Service Wrapper

Code:
curl -L -k http://github.com/elasticsearch/elasticsearch-servicewrapper/tarball/master | tar -xz
mv *servicewrapper*/service elasticsearch/bin/
elasticsearch/bin/service/elasticsearch install
ln -s `readlink -f elasticsearch/bin/service/elasticsearch` /usr/local/bin/rcelasticsearch
rcelasticsearch start

Assuming everything was done correctly, you should see the following output.

Code:
rcelasticsearch start
Starting ElasticSearch...
Waiting for ElasticSearch......
running: PID: xxxxx

Basic Configuration

You should do some basic configuration of Elasticsearch before installing the addon in XenForo.


1) Open up /elasticsearch/config/elasticsearch.yml and on line 32 edit

Code:
# cluster.name: elasticsearch

To

Code:
cluster.name: PUT-SOMETHING-UNIQUE-HERE

on line 199 edit

Code:
# network.host: 192.168.0.1

to

Code:
network.host: 127.0.0.1

On line 211 edit

Code:
# http.port: 9200

to

Code:
http.port: 9200

Save and Close


2) Open up /elasticsearch/bin/service/elasticsearch.conf on line 2 edit

Code:
set.default.ES_HEAP_SIZE=1024

To a number suitable for the size of your forum.

I reccomend approximately 1 GB for the HEAP_SIZE per 1 million posts on your forum.

1 Million Posts: 1024
2 Million Posts: 2048
3 Million Posts: 3072
4 Million Posts: 4096
etc

This will not mean the service will use all that available memory, however it will have it at its disposal if required.

So for example a 3 Million Post forum would edit

Code:
set.default.ES_HEAP_SIZE=1024

to

Code:
set.default.ES_HEAP_SIZE=3072



Save and Exit.


3) Optional - Move the Elasticsearch data directory.

Your linux install may be configured in such a way that your install partition is only a few Gb in size, and placing a large Elasticsearch index there is not ideal.

In which case you will want to move the index directory to a different, larger, location (in this example /var/elasticsearch)

Code:
cd /var
mkdir elasticsearch

Open up /elasticsearch/config/elasticsearch.yml on line 143 edit

Code:
# path.data: /path/to/data

to

Code:
path.data: /var/elasticsearch

Save and Exit

4) Restart the Elasticsearch Service

In SSH type

Code:
rcelasticsearch restart

You should get the following output

Code:
rcelasticsearch restart
Stopping ElasticSearch...
Stopped ElasticSearch.
Starting ElasticSearch...
Waiting for ElasticSearch......
running: PID: xxxxx

Elasticsearch is now runing with your updated config.



Install the XenForo Enhanced Search Addon

1) Turn your board off into maintainance mode*

2) Download the addon from your customer area at http://xenforo.com/customers/

3) Follow the instructions found at http://xenforo.com/help/enhanced-search/

4) Wait for your indexes to be rebuilt

5) Open your board.

6) Install the index pre-warmer.

As of 0.90.0 Beta an index pre-warmer is available. This keeps your search index "warm" in active memory so when a search is done, the access time latency is highly reduced.

Installing this is simple, in SSH simply run the following replacing the *INDEX NAME* with the name of your ES index.

Code:
curl -XPUT localhost:9200/*INDEX NAME*/_warmer/warmer_1 -d '{
    "query" : {
        "match_all" : {}
    }
}'

You should have the following returned

Code:
{"ok":true,"acknowledged":true}


*You may leave your board open during the re-index process.

Congratulations. Your board should now be running XenForo Enhanced Search.
 
Last edited:
At the java download page from sun you can also find detailled information about how to upgrade, install or remove the rpms.

As an example, I installed the 32bit rpm, later I installed the 64bit java and of course got problems: not only the 32bit did not work, the 64bit java was still in place but unuseable. The manual from sun told me how to deinstall both 64bit and 32bit rpms and how to reinstall the 64bit java.

I wonder how I could update the latest java with YUM. That would be the perfect solution.
 
Code:
# yum list \*java\* | grep "jdk"
java-1.6.0-openjdk.x86_64                     1:1.6.0.0-3.1.13.1.el6_5  updates
java-1.6.0-openjdk-demo.x86_64                1:1.6.0.0-3.1.13.1.el6_5  updates
java-1.6.0-openjdk-devel.x86_64               1:1.6.0.0-3.1.13.1.el6_5  updates
java-1.6.0-openjdk-javadoc.x86_64             1:1.6.0.0-3.1.13.1.el6_5  updates
java-1.6.0-openjdk-src.x86_64                 1:1.6.0.0-3.1.13.1.el6_5  updates
java-1.7.0-openjdk.x86_64                     1:1.7.0.51-2.4.4.1.el6_5  updates
java-1.7.0-openjdk-demo.x86_64                1:1.7.0.51-2.4.4.1.el6_5  updates
java-1.7.0-openjdk-devel.x86_64               1:1.7.0.51-2.4.4.1.el6_5  updates
java-1.7.0-openjdk-javadoc.noarch             1:1.7.0.51-2.4.4.1.el6_5  updates
java-1.7.0-openjdk-src.x86_64                 1:1.7.0.51-2.4.4.1.el6_5  updates
ldapjdk-javadoc.x86_64                        4.18-6.el6                base
 
Anyone using #yum install elasticsearch?

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-repositories.html

Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package elasticsearch.noarch 0:1.1.1-1 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
elasticsearch noarch 1.1.1-1 elasticsearch-1.1 18 M

Transaction Summary
================================================================================
Install 1 Package(s)

Total download size: 18 M
Installed size: 21 M
Is this ok [y/N]:
 
Last edited:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-repositories.html

We also have repositories available for APT and YUM based distributions.

We have split the major versions in separate urls to avoid accidental upgrades across major version. For all 0.90.x releases use 0.90 as version number, for 1.0.x use 1.0, etc.

apt
Download and install the Public Signing Key

wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add -
Add the following to your /etc/apt/sources.list to enable the repository

deb http://packages.elasticsearch.org/elasticsearch/0.90/debian stable main
Run apt-get update and the repository is ready for use.

yum
Download and install the Public Signing Key

rpm --import http://packages.elasticsearch.org/GPG-KEY-elasticsearch
Add the following in your /etc/yum.repos.d/ directory

[elasticsearch-0.90]
name=Elasticsearch repository for0.90.x packages
baseurl=http://packages.elasticsearch.org/elasticsearch/0.90/centos
gpgcheck=1
gpgkey=http://packages.elasticsearch.org/GPG-KEY-elasticsearch
enabled=1
And your repository is ready for use.
 
Thanks for the heads up. What's exactly bad about the package? I did not find more information about that.

yum install elasticsearch is far more easier than to download both elasticsearch and java, install both of them with different commands and repeat that for each update. It would help very much to make the setup guide easier, hence also help selling the product.
 
Last edited:
Thanks @Floren (y)
Code:
Installed Packages
Name        : elasticsearch
Arch        : x86_64
Version     : 1.1.1
Release     : 1.el6
Size        : 20 M
Repo        : installed
From repo   : axivo
Summary     : Distributed RESTful full-text search engine
URL         : http://www.elasticsearch.org
License     : ASL 2.0
Description : ElasticSearch is a distributed, scalable, multitenant-capable full-text search
            : engine with a RESTful web interface and schema-free JSON documents. It uses
            : Lucene and tries to make all features of it available through the JSON and Java
            : API. It supports real-time GET requests, faceting and percolating, which can be
            : useful for notifying if new documents match for registered queries.
 
@MattW, pretty easy to setup ehh? :) No wrappers and other mumbo-jumbo, just a clean and efficient package. I'll post a tutorial soon so people know how to adjust everything. For now, look at:
/etc/elasticsearch
/etc/sysconfig/elasticsearch
/etc/security/limits.d/60-elasticsearch.conf

Can't be easier than that to set everything. :)
I did some major cleanup on the config files... there was a lot of confusion in there, as they are samples. My RPM will get you up and running right away, without any tweaks, I provided these advanced settings for users who like to tweak their box to the bone.
 
Last edited:
Very easy to set up. I had some issues after removing the old elasticsearch rpm as it left the old folders in place with the old uid.gid of the previous elasticsearch user, so had to set them back to elasticsearch

Code:
[root@host sysconfig]# log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /var/log/elasticsearch/mattwservices.log (Permission denied)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:142)
        at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
        at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
        at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
        at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
        at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:440)
        at org.elasticsearch.common.logging.log4j.LogConfigurator.configure(LogConfigurator.java:111)
        at org.elasticsearch.bootstrap.Bootstrap.setupLogging(Bootstrap.java:92)
        at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:176)
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
log4j:ERROR Either File or DatePattern options are not set for appender [file].
Code:
[root@host elasticsearch]# ls -hl
total 44K
-rw-r--r-- 1 elasticsearch elasticsearch    0 May  4 10:53 mattwservices.index.indexing.slow.log
-rw-r--r-- 1           497           494    0 Feb 22 12:34 mattwservices_index_indexing_slowlog.log
-rw-r--r-- 1 elasticsearch elasticsearch    0 May  4 10:53 mattwservices.index.search.slow.log
-rw-r--r-- 1           497           494    0 Feb 22 12:34 mattwservices_index_search_slowlog.log
-rw-r--r-- 1           497           494  352 May  4 10:47 mattwservices.log
-rw-r--r-- 1           497           494 6.1K Feb 22 19:11 mattwservices.log.2014-02-22
-rw-r--r-- 1           497           494 1.9K Feb 25 21:08 mattwservices.log.2014-02-25
-rw-r--r-- 1           497           494  132 Feb 26 15:57 mattwservices.log.2014-02-26
-rw-r--r-- 1           497           494  272 Mar 19 20:06 mattwservices.log.2014-03-19
-rw-r--r-- 1           497           494  124 Mar 20 19:45 mattwservices.log.2014-03-20
-rw-r--r-- 1           497           494  133 Mar 23 19:30 mattwservices.log.2014-03-23
-rw-r--r-- 1           497           494  127 Mar 24 11:52 mattwservices.log.2014-03-24
-rw-r--r-- 1           497           494  388 Apr 22 07:21 mattwservices.log.2014-04-22
-rw-r--r-- 1           497           494  135 Apr 28 05:59 mattwservices.log.2014-04-28
and
Code:
[2014-05-04 10:47:47,772][INFO ][node                     ] [Bennet du Paris] stopping ...
[2014-05-04 10:47:47,792][INFO ][node                     ] [Bennet du Paris] stopped
[2014-05-04 10:47:47,792][INFO ][node                     ] [Bennet du Paris] closing ...
[2014-05-04 10:47:47,802][INFO ][node                     ] [Bennet du Paris] closed
[2014-05-04 10:55:01,224][INFO ][node                     ] [MWS Host] version[1.1.1], pid[14754], build[f1585f0/2014-04-16T14:27:12Z]
[2014-05-04 10:55:01,225][INFO ][node                     ] [MWS Host] initializing ...
[2014-05-04 10:55:01,227][INFO ][plugins                  ] [MWS Host] loaded [], sites []
[2014-05-04 10:55:01,247][ERROR][bootstrap                ] {1.1.1}: Initialization Failed ...
- ElasticsearchIllegalStateException[Failed to obtain node lock, is the following location writable?: [/var/lib/elasticsearch/mattwservices]]
        IOException[failed to obtain lock on /var/lib/elasticsearch/mattwservices/nodes/49]
                IOException[Cannot create directory: /var/lib/elasticsearch/mattwservices/nodes/49]
[2014-05-04 10:58:05,807][INFO ][node                     ] [MWS Host] version[1.1.1], pid[15508], build[f1585f0/2014-04-16T14:27:12Z]
[2014-05-04 10:58:05,807][INFO ][node                     ] [MWS Host] initializing ...
[2014-05-04 10:58:05,810][INFO ][plugins                  ] [MWS Host] loaded [], sites []
[2014-05-04 10:58:05,830][ERROR][bootstrap                ] {1.1.1}: Initialization Failed ...
- ElasticsearchIllegalStateException[Failed to obtain node lock, is the following location writable?: [/var/lib/elasticsearch/mattwservices]]
        IOException[failed to obtain lock on /var/lib/elasticsearch/mattwservices/nodes/49]
                IOException[Cannot create directory: /var/lib/elasticsearch/mattwservices/nodes/49]
Code:
[root@host elasticsearch]# ls -al
total 12
drwxr-xr-x  3 elasticsearch root 4096 Apr 18 23:15 .
drwxr-xr-x 24 root          root 4096 Apr 22 11:41 ..
drwxr-xr-x  3           497  494 4096 Feb 22 12:34 mattwservices
[root@host elasticsearch]# chown -R elasticsearch.elasticsearch *
[root@host elasticsearch]# ls -al
total 12
drwxr-xr-x  3 elasticsearch root          4096 Apr 18 23:15 .
drwxr-xr-x 24 root          root          4096 Apr 22 11:41 ..
drwxr-xr-x  3 elasticsearch elasticsearch 4096 Feb 22 12:34 mattwservices
[root@host elasticsearch]#
but other than that, it's up and running.

I also add into the yaml config
Code:
network.tcp.block: true
node.local: true
because it's on a single localhost node
 
@MattW, I posted a tutorial. No more issues with the configuration. :)
http://xenforo.com/community/resources/elasticsearch-setup.3155/
Very easy to set up. I had some issues after removing the old elasticsearch rpm
Was that the Elasticsearch RPM they provide on their site? It is a disaster, so many non-standard things on it.

You don't need the TCP block or to define a local node if you serve your index from localhost.
Edit: Just saw this now.. I always use a column separator (:, not .) to define the user and group:
# chown elasticsearch:elasticsearch *
I had no idea you could use a dot, heh. :giggle:
 
Last edited:
@MattW, I posted a tutorial. No more issues with the configuration. :)
http://xenforo.com/community/resources/elasticsearch-setup.3155/

Was that the Elasticsearch RPM they provide on their site? It is a disaster, so many non-standard things on it.
Yes, it was the 1.0.0 rpm from their site.
Code:
[root@host init.d]# rpm -qa | grep elasticsearch
elasticsearch-1.0.0-1.noarch
You don't need the TCP block or to define a local node if you serve your index from localhost.
Edit: Just saw this now.. I always use a column separator (:, not .) to define the user and group:

I had no idea you could use a dot, heh. :giggle:
I've always used a . for the separator, mainly because when I started working where I am now, we had mostly SCO boxes (still have a few....but they fail PCI)
Code:
GIBTNAC3# uname -X

System = GIBTNAC3
Node = GIBTNAC3
Release = 3.2v4.2
KernelID = 93/04/28
Machine = Pentium
BusType = ISA
Serial = 2FI067499
Users = 16-user
OEM# = 0
Origin# = 1
NumCPU = 1
 
@MattW did you install this on a centminmod box?
Did you have to remove the old installation from this guide?

@Floren
Thanks for the tutorial. I have the samething about what to do with the current elasticsearch installed with this guide.
 
@MattW did you install this on a centminmod box?
Did you have to remove the old installation from this guide?

@Floren
Thanks for the tutorial. I have the samething about what to do with the current elasticsearch installed with this guide.
Yes, this was on a centminmod box. I didn't install it on this server using this guide. When I initially configured the server, it was installed using the rpm provided by elasticsearch and installed that way. I still had to uninstall it though to use the new repo version.
 
I'm going to post an update to the tutorial, teaching you how to properly uninstall the official RPM. Is a real pain to clean everything, due to the fact their RPM is not built properly.
I have removed and installed elasticsearch according your tutorial using Axivo package.
Everything seems painless. Thanks a lot for your work.

It however, still shows as I have an old version?
curl http://localhost:9200
{
"ok" : true,
"status" : 200,
"name" : "Maximoff, Pietro",
"version" : {
"number" : "0.90.9",
"build_hash" : "a968646da4b6a2d9d8bca9e51e92597fe64e8d1a",
"build_timestamp" : "2013-12-23T10:35:28Z",
"build_snapshot" : false,
"lucene_version" : "4.6"
},
"tagline" : "You Know, for Search"
}
 
Mines:
Code:
# curl localhost:9200
{
  "status" : 200,
  "name" : "Lovin RHEL",
  "version" : {
    "number" : "1.1.1",
    "build_hash" : "f1585f096d3f3985e73456debdc1a0745f512bbc",
    "build_timestamp" : "2014-04-16T14:27:12Z",
    "build_snapshot" : false,
    "lucene_version" : "4.7"
  },
  "tagline" : "You Know, for Search"
}
I suggest you to ask your questions here:
https://www.axivo.com/forums/software.8/
 
Top Bottom