How to: Basic Elasticsearch installation. (RHEL/SUSE)

Slavik

XenForo moderator
Staff member
Basic Elasticsearch Installation (RHEL / SUSE)

@Floren has an amazing repo for RHEL 6 and 7, the Elasticsearch RPM he provides is perfectly set up and currently I suggest using it.

The old manual setup guide can be found below.

Step 1) Install the Axivo Repo: https://www.axivo.com/resources/repository-setup.1/

Step 2) Install ElasticSearch: https://www.axivo.com/resources/elasticsearch-setup.11/




This guide is provided to show how to do a basic (vanilla get up and go) install of Elasticsearch (0.90.0 Beta 1), the Elasticsearch Service Wrapper and the required Java Runtime Environment (JRE) (1.7.0_17) on RHEL / SUSE. This guide will not cover running a dedicated Elasticsearch user.

For Debian/Ubuntu users, a guide can be found here.

This guide assumes the user has basic knowledge of SSH and prior to starting the steps below has logged in as root. This guide also assumes the user does not currently have any JRE installed. You can check if you have JRE installed by typing

Code:
java -version

As of writing, the current file locations for JRE are as follows:

32 bit
Code:
http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-i586.rpm

64 bit
Code:
http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-x64.rpm

The guide will be shown using the 64 bit install, however if you are using a 32 bit system, change the file names as appropriate.

Please note, whilst this is a simple and easy setup, I take no responsibility for any damages or losses that may occur to your system by following the steps below. If you are unsure at any stage, please ask for assistance or seek the help of a qualified Linux Systems Administrator.

Installing the JRE
Type the following commands into your SSH terminal.
Code:
cd /tmp
wget http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-x64.rpm
rpm -ivh jre-7u17-linux-x64.rpm
java -version

Assuming everything was done correctly, you should get the following output.

Code:
# java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode)

Install Elasticsearch

Code:
cd /
curl -L -O -k https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.0.Beta1.zip
unzip elasticsearch-0.90.0.Beta1.zip
mv elasticsearch-0.90.0.Beta1 elasticsearch

Install the Elasticsearch Service Wrapper

Code:
curl -L -k http://github.com/elasticsearch/elasticsearch-servicewrapper/tarball/master | tar -xz
mv *servicewrapper*/service elasticsearch/bin/
elasticsearch/bin/service/elasticsearch install
ln -s `readlink -f elasticsearch/bin/service/elasticsearch` /usr/local/bin/rcelasticsearch
rcelasticsearch start

Assuming everything was done correctly, you should see the following output.

Code:
rcelasticsearch start
Starting ElasticSearch...
Waiting for ElasticSearch......
running: PID: xxxxx

Basic Configuration

You should do some basic configuration of Elasticsearch before installing the addon in XenForo.


1) Open up /elasticsearch/config/elasticsearch.yml and on line 32 edit

Code:
# cluster.name: elasticsearch

To

Code:
cluster.name: PUT-SOMETHING-UNIQUE-HERE

on line 199 edit

Code:
# network.host: 192.168.0.1

to

Code:
network.host: 127.0.0.1

On line 211 edit

Code:
# http.port: 9200

to

Code:
http.port: 9200

Save and Close


2) Open up /elasticsearch/bin/service/elasticsearch.conf on line 2 edit

Code:
set.default.ES_HEAP_SIZE=1024

To a number suitable for the size of your forum.

I reccomend approximately 1 GB for the HEAP_SIZE per 1 million posts on your forum.

1 Million Posts: 1024
2 Million Posts: 2048
3 Million Posts: 3072
4 Million Posts: 4096
etc

This will not mean the service will use all that available memory, however it will have it at its disposal if required.

So for example a 3 Million Post forum would edit

Code:
set.default.ES_HEAP_SIZE=1024

to

Code:
set.default.ES_HEAP_SIZE=3072



Save and Exit.


3) Optional - Move the Elasticsearch data directory.

Your linux install may be configured in such a way that your install partition is only a few Gb in size, and placing a large Elasticsearch index there is not ideal.

In which case you will want to move the index directory to a different, larger, location (in this example /var/elasticsearch)

Code:
cd /var
mkdir elasticsearch

Open up /elasticsearch/config/elasticsearch.yml on line 143 edit

Code:
# path.data: /path/to/data

to

Code:
path.data: /var/elasticsearch

Save and Exit

4) Restart the Elasticsearch Service

In SSH type

Code:
rcelasticsearch restart

You should get the following output

Code:
rcelasticsearch restart
Stopping ElasticSearch...
Stopped ElasticSearch.
Starting ElasticSearch...
Waiting for ElasticSearch......
running: PID: xxxxx

Elasticsearch is now runing with your updated config.



Install the XenForo Enhanced Search Addon

1) Turn your board off into maintainance mode*

2) Download the addon from your customer area at http://xenforo.com/customers/

3) Follow the instructions found at http://xenforo.com/help/enhanced-search/

4) Wait for your indexes to be rebuilt

5) Open your board.

6) Install the index pre-warmer.

As of 0.90.0 Beta an index pre-warmer is available. This keeps your search index "warm" in active memory so when a search is done, the access time latency is highly reduced.

Installing this is simple, in SSH simply run the following replacing the *INDEX NAME* with the name of your ES index.

Code:
curl -XPUT localhost:9200/*INDEX NAME*/_warmer/warmer_1 -d '{
    "query" : {
        "match_all" : {}
    }
}'

You should have the following returned

Code:
{"ok":true,"acknowledged":true}


*You may leave your board open during the re-index process.

Congratulations. Your board should now be running XenForo Enhanced Search.
 
Last edited:
I've just finished removing the initial setup as per the original post in this guide, and moved to @Floren 's repo install:

Code:
root@host [/etc/init.d]# yum info elasticsearch
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Installed Packages
Name        : elasticsearch
Arch        : x86_64
Version     : 1.1.1
Release     : 1.el6
Size        : 20 M
Repo        : installed
From repo   : axivo
Summary     : Distributed RESTful full-text search engine
URL         : http://www.elasticsearch.org
License     : ASL 2.0
Description : ElasticSearch is a distributed, scalable, multitenant-capable full-text search
            : engine with a RESTful web interface and schema-free JSON documents. It uses
            : Lucene and tries to make all features of it available through the JSON and Java
            : API. It supports real-time GET requests, faceting and percolating, which can be
            : useful for notifying if new documents match for registered queries.

root@host [/etc/init.d]#

From start to finish:

Code:
  527  rcelasticsearch stop
  528  ps -ef
  529  htop
  530  java --version
  531  java -version
  532  rpm -qa | grep jre
  533  rpm -e jre-1.7.0_51-fcs.x86_64
  534  java -version
  535  cd /
  536  ls
  537  cd /root/
  538  ls
  539  ls -al
  540  rm -rf elasticsearch/
  541  rm elasticsearch-0.90.9.zip
  542  cd /usr/local/bin/
  543  ls
  544  ls -al
  545  rm rcelasticsearch
  546  cd /etc/init.d/
  547  ls
  548  rm elasticsearch
  549  ls
  550  yum --enablerepo=axivo install elasticsearch
  551  java -version
  552  sed -i 's|^#cluster.name.*|cluster.name: ironbubble|' /etc/elasticsearch/elasticsearch.yml
  553  chkconfig elasticsearch on
  554  service elasticsearch start
 
As a first step, the tutorial might be rewritten from

This guide also assumes the user does not currently have any JRE installed. You can check if you have JRE installed by typing

Code:
java -version
As of writing, the current file locations for JRE are as follows:

32 bit
Code:
http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-i586.rpm
64 bit
Code:
http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-x64.rpm
The guide will be shown using the 64 bit install, however if you are using a 32 bit system, change the file names as appropriate.

Please note, whilst this is a simple and easy setup, I take no responsibility for any damages or losses that may occur to your system by following the steps below. If you are unsure at any stage, please ask for assistance or seek the help of a qualified Linux Systems Administrator.

Installing the JRE
Type the following commands into your SSH terminal.
Code:
cd /tmp
wget http://download.oracle.com/otn-pub/java/jdk/7u17-b02/jre-7u17-linux-x64.rpm
rpm -ivh jre-7u17-linux-x64.rpm
java -version

Assuming everything was done correctly, you should get the following output.

Code:
# java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17)
Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode)

to a simple

Code:
Removing old java installs
Step 1: Display all installed java releases, installed with # rpm .....
# rpm -qa | grep "jre"

Step 2:Remove each jra-..... releases with
# rpm -e jre-1.7.0_45-fcs

This installs Java with YUM when it is not already installed:
# yum -y install java-1.7.0-openjdk

This removes "rpm java installs" and installs java with yum.
 
Last edited:
java 7 required
Elasticsearch now requires Java 7 and will no longer work with Java 6. We recommend using Oracle’s JDK 7u55 or JDK 7u25. Avoid any of the updates in-between as they contain a nasty bug which can cause index corruption.
We implemented this requirement from the beginning into AXIVO rpms, since it was the logical choice to do.
 
I don't know what I'm doing wrong but I cannot get this to work properly.
I had previously used this guide to install on a CentOS 6.5 x64 Digital Ocean droplet. The droplet was hacked and shut down.
I've tried installing elasticsearch under a (presumably) more secure Centmin Mod install and without it. Every time I get this "WARNING: Elasticsearch may have failed to start."
I've tried the exact versions of JRE and ES in the guide, I've tried the absolute latest versions as well. What am I doing wrong?
 
I got the elasticsearch server to start but the pre-warmer does not work for me...
I get the following
Code:
{"error":"IndexMissingException[[myindex] missing]","status":404}
I set the index name in the XF ACP and it indexes fine but this doesn't work and the search function also does not work. Is there a place I need to specify the index name on the ES server?
 
I don't know what I'm doing wrong but I cannot get this to work properly.
I had previously used this guide to install on a CentOS 6.5 x64 Digital Ocean droplet. The droplet was hacked and shut down.
I've tried installing elasticsearch under a (presumably) more secure Centmin Mod install and without it. Every time I get this "WARNING: Elasticsearch may have failed to start."
I've tried the exact versions of JRE and ES in the guide, I've tried the absolute latest versions as well. What am I doing wrong?
https://www.axivo.com/resources/elasticsearch-setup.11/
 
  • Like
Reactions: rdn
Thanks everyone - I removed old JRE, removed old elastic search, reinstalled JRE from yum and installed the Axivo build in about 5 minutes thanks to the helpful posts on this page. All working perfectly.
 
Last edited:
If you use the former setting "weighted relevance" (which works for you) and then switch to another elasticsearch installation which does not support it, your users will get a "The search could not be completed. Please try again later." message.

To enable searching agian all you have to do is to disable this setting in acp>search.
 
Thanks everyone - I removed old JRE, removed old elastic search, reinstalled JRE from yum and installed the Axivo build in about 5 minutes thanks to the helpful posts on this page. All working perfectly.
Repos are wonderful compared to manually download, deinstall and install all the different modules needed for enhanced search.
 
Crap, I wish I had checked the latest posts in this thread... that repo install looks good.

I just had to move my ES database to a new server and followed the old instructions in the OP, except with the latest versions of the JRE and ES.

Would it be worth it to go back, un-install the old version and start again with the repo? Will it be easier to maintain in the long run, updating ES versions, etc.?

Also, with the install in the OP, ES is running as root, which doesn't seem smart. Can I change that?
 
Do we have a separate thread for configuring or tweaking ES? I got an email from our host yesterday saying that our old version of ES was hacked, and I had to scramble to get a new one working. I went with ES 1.1.2, as versions 1.2.0 and beyond need Java 7, and I didn't have time to screw around with that mess yet. Well...it's running fine, except that after I dumped and rebuilt the indexes, Java is now using up 2½ times as much memory.

Not sure if I made a configuration error, or if the new version is an even worse memory hog. But I needed to get it back online in a hurry, so hopefully I can do my triage over the weekend.

Also I noticed something odd, but I need to check my add-ons first. When I hit the server to check the mappings, the mapping was larger for our private test forum than it was for the live forum. Both of these mappings are "optimized" by XF's Enhanced Search add-on.

BTW, we are on Ubuntu and such, so the automated options here will not work for us. I might, however, compare the config (yml) file to mine and see what is different.
 
I've uninstalled the RPM version of ElasticSearch and re-installed @Floren's AXIVO repository.

ElasticSearch seems to be running fine:
Code:
$ curl -XGET 'http://127.0.0.1:9200'
{
  "status" : 200,
  "name" : "Wasp",
  "version" : {
    "number" : "1.3.0",
    "build_hash" : "1265b1454eee7725a6918f57415c480028700fb4",
    "build_timestamp" : "2014-07-23T13:46:36Z",
    "build_snapshot" : false,
    "lucene_version" : "4.9"
  },
  "tagline" : "You Know, for Search"
}
But I cannot get XenForo to connect to it at all; I get nothing but "Elasticsearch returned no response..." errors in my XF error log. This is all running on localhost and worked perfectly before. Same port, same everything...

I'm out of ideas. Can anyone help?
 
Back
Top Bottom