• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Quick question before buying XenForo

Ryan_

Active member
#1
Hi everyone! I recently decided to buy XenForo and import my data from my vBulletin 4 forum after testing out the demos - excellent software for sure! I just had a quick question I was hoping you guys could help me with:

I want to set up the XenForo Enhanced Search, but I am a little confused on how to set up elasticsearch:

* "Download":http://www.elasticsearch.org/download and unzip the ElasticSearch official distribution.
* Run @bin/elasticsearch -f@ on unix, or @bin/elasticsearch.bat@ on windows.
* Run @curl -X GET http://localhost:9200/@.
* Start more servers ...

I understand the first part. The 2nd part seems easy enough for my Linux VPS. The 3rd part, is it literally localhost or are you supposed to replace that with something? And what do they mean by the "start more servers" part? Also, after that they give me an example for indexing:

h3. Indexing

Lets try and index some twitter like information. First, lets create a twitter user, and add some tweets (the @twitter@ index will be created automatically):

<pre>
curl -XPUT 'http://localhost:9200/twitter/user/kimchy' -d '{ "name" : "Shay Banon" }'

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?"
}'

curl -XPUT 'http://localhost:9200/twitter/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?"
}'
</pre>

Now, lets see if the information was added by GETting it:

<pre>
curl -XGET 'http://localhost:9200/twitter/user/kimchy?pretty=true'
curl -XGET 'http://localhost:9200/twitter/tweet/1?pretty=true'
curl -XGET 'http://localhost:9200/twitter/tweet/2?pretty=true'
</pre>

h3. Searching

Mmm search..., shouldn't it be elastic?
Lets find all the tweets that @kimchy@ posted:

<pre>
curl -XGET 'http://localhost:9200/twitter/tweet/_search?q=user:kimchy&pretty=true'
</pre>

We can also use the JSON query language ElasticSearch provides instead of a query string:

<pre>
curl -XGET 'http://localhost:9200/twitter/tweet/_search?pretty=true' -d '
{
"query" : {
"text" : { "user": "kimchy" }
}
}'
</pre>

Just for kicks, lets get all the documents stored (we should see the user as well):

<pre>
curl -XGET 'http://localhost:9200/twitter/_search?pretty=true' -d '
{
"query" : {
"matchAll" : {}
}
}'
</pre>

We can also do range search (the @postDate@ was automatically identified as date)

<pre>
curl -XGET 'http://localhost:9200/twitter/_search?pretty=true' -d '
{
"query" : {
"range" : {
"postDate" : { "from" : "2009-11-15T13:00:00", "to" : "2009-11-15T14:00:00" }
}
}
}'
</pre>

There are many more options to perform search, after all, its a search product no? All the familiar Lucene queries are available through the JSON query language, or through the query parser.

h3. Multi Tenant - Indices and Types

Maan, that twitter index might get big (in this case, index size == valuation). Lets see if we can structure our twitter system a bit differently in order to support such large amount of data.

ElasticSearch support multiple indices, as well as multiple types per index. In the previous example we used an index called @twitter@, with two types, @User@ and @Tweet@.

Another way to define our simple twitter system is to have a different index per user (though note that an index has an overhead). Here is the indexing curl's in this case:

<pre>
curl -XPUT 'http://localhost:9200/kimchy/info/1' -d '{ "name" : "Shay Banon" }'

curl -XPUT 'http://localhost:9200/kimchy/tweet/1' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T13:12:00",
"message": "Trying out Elastic Search, so far so good?"
}'

curl -XPUT 'http://localhost:9200/kimchy/tweet/2' -d '
{
"user": "kimchy",
"postDate": "2009-11-15T14:12:12",
"message": "Another tweet, will it be indexed?"
}'
</pre>

The above index information into the @kimchy@ index, with two types, @info@ and @Tweet@. Each user will get his own special index.

Complete control on the index level is allowed. As an example, in the above case, we would want to change from the default 5 shards with 1 replica per index, to only 1 shard with 1 replica per index (== per twitter user). Here is how this can be done (the configuration can be in yaml as well):

<pre>
curl -XPUT http://localhost:9200/another_user/ -d '
{
"index" : {
"numberOfShards" : 1,
"numberOfReplicas" : 1
}
}'
</pre>

Search (and similar operations) are multi index aware. This means that we can easily search on more than one
index (twitter user), for example:

<pre>
curl -XGET 'http://localhost:9200/kimchy,another_user/_search?pretty=true' -d '
{
"query" : {
"matchAll" : {}
}
}'
</pre>

Or on all the indices:

<pre>
curl -XGET 'http://localhost:9200/_search?pretty=true' -d '
{
"query" : {
"matchAll" : {}
}
}'
</pre>

{One liner teaser}: And the cool part about that? You can easily search on multiple twitter users (indices), with different boost levels per user (index), making social search so much simpler (results from my friends rank higher than results from my friends friends).

h3. Distributed, Highly Available

Lets face it, things will fail....

ElasticSearch is a highly available and distributed search engine. Each index is broken down into shards, and each shard can have one or more replica. By default, an index is created with 5 shards and 1 replica per shard (5/1). There are many topologies that can be used, including 1/10 (improve search performance), or 20/1 (improve indexing performance, with search executed in a map reduce fashion across shards).

In order to play with Elastic Search distributed nature, simply bring more nodes up and shut down nodes. The system will continue to serve requests (make sure you use the correct http port) with the latest data indexed.

h3. Where to go from here?

We have just covered a very small portion of what ElasticSearch is all about. For more information, please refer to: .

h3. Building from Source

ElasticSearch uses "Maven":http://maven.apache.org for its build system.

In order to create a distribution, simply run the @mvn clean package -DskipTests@ command in the cloned directory.

The distribution will be created under @target/releases@.

h1. License

<pre>
This software is licensed under the Apache 2 license, quoted below.

Copyright 2009-2013 Shay Banon and ElasticSearch <http://www.elasticsearch.org>

Licensed under the Apache License, Version 2.0 (the "License"); you may not
use this file except in compliance with the License. You may obtain a copy of
the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations under
the License.
</pre>
Is any of that required to be done, or is it just an example? As I am not very familiar with SSH, I prefer doing a lot of things via other methods when applicable. And once all that is done, you just install the Enhanced Search add-on and the files that go with it and it takes affect immediately, correct?
 

AndyB

Well-known member
#3
I want to set up the XenForo Enhanced Search, but I am a little confused on how to set up elasticsearch:
Setting up ElasticSearch is super easy and does not require leaving the Admin CP.

What you quoted is advanced stuff for those that want to create a custom ElasticSearch index.
 

Ryan_

Active member
#4

Slavik

XenForo moderator
Staff member
#5
Thanks Brogan, I found this thread and it seems very helpful: http://xenforo.com/community/threads/how-to-basic-elasticsearch-installation-debian-ubuntu.26163/



Oh okay, good to know. Don't you need to have root SSH access to install it though, as explained via the above linked thread?

All you need to do is install and start the Elasticsearch service. (Optionally configure it also)

Then in XenForo, install the addon and enable the search to use it, then rebuild the cache. The addon will create the search schemas for you.
 

AndyB

Well-known member
#6
Don't you need to have root SSH access to install it though, as explained via the above linked thread?
Correct if you want to install EleasticSearch on your server you would need ssh and root.

In my case I just emailed my web hosting company and asked them to install ElasticSearch onto my dedicated server.
 

Ryan_

Active member
#7
Okay, I understand how to do it now. Install it on the server, install the XenForo addon and follow the instructions for it, and then it's working. Thanks a lot for your help guys! Can't wait to purchase XenForo soon.
 

Brogan

XenForo moderator
Staff member
#10
I disagree, you will definitely regret it.

Whereas before you had hours of free time, now you will spend those hours styling, tweaking the style again, checking out add-ons and setting them up, fiddling with the style again, playing with the features, posting more, styling, and so on :D
 

MattW

Well-known member
#11
I disagree, you will definitely regret it.

Whereas before you had hours of free time, now you will spend those hours styling, tweaking the style again, checking out add-ons and setting them up, fiddling with the style again, playing with the features, posting more, styling, and so on :D
and spending just as much time on here as you do on your own site!
 

Ryan_

Active member
#13
I disagree, you will definitely regret it.

Whereas before you had hours of free time, now you will spend those hours styling, tweaking the style again, checking out add-ons and setting them up, fiddling with the style again, playing with the features, posting more, styling, and so on :D
Looks like I'll need more hours of the day to continue to have a life outside the internet then, eh? :p Nah, who needs a life outside the internet when you have a XenForo forum, filled with such wonders as you just described - what more could one ask for in their life!

Your post is edited for correctness. :p
I think I'll stick with equal amounts of time on both, as my members would begin to miss me otherwise. :D
 

Ryan_

Active member
#14
Oh and I thought up a couple more questions for you guys:

1. Can anyone recommend a good add-on or way to daily backup my database and my files, with instructions if needed?
2. Does VAT apply for someone in the US? I don't think it does thanks to my research, I just want to make sure.
 

Ryan_

Active member
#17
VAT does not apply in the US.
Thanks, glad someone confirmed that!
I was looking at that one, and the only question I had is this:

For my own use, I backup several databases each night, so the perl script is creating an array of file names, and using a foreach loop. If you only have a single database to back up, you can remove the foreach loop, and set the array to be a single variable.
How would I remove the foreach loop and set the array to be a single variable, since I will only be using one XenForo site and therefore one database/file system? I know I haven't bought XenForo yet, but it'd be nice to know before I actually buy it and download the add-on.
 

Jeremy

Well-known member
#18
Without looking at the add-on, I can't really help. But MattW is very active and very helpful. I'm sure he'll help you out, if not, the community here is knowledgeable and can help.
 

MattW

Well-known member
#19
How would I remove the foreach loop and set the array to be a single variable, since I will only be using one XenForo site and therefore one database/file system? I know I haven't bought XenForo yet, but it'd be nice to know before I actually buy it and download the add-on.
All you need to do is change the array of files to a single variable:

Code:
$file = "$dir/z22seforum.$today.sql.bz2"
then, remove the foreach loop from the $ftp->put command
Code:
        $ftp->put($file)
          or warn "Upload failed ", $ftp->message;
 

Ryan_

Active member
#20
All you need to do is change the array of files to a single variable:

Code:
$file = "$dir/z22seforum.$today.sql.bz2"
then, remove the foreach loop from the $ftp->put command
Code:
        $ftp->put($file)
          or warn "Upload failed ", $ftp->message;
I'm assuming in that case you DON'T need to do the same thing for $ftp->delete and the line under "# Array of Old Database Files to be Deleted", correct? Also, do you replace http://prntscr.com/1gs5sa << those with the actual information?