Would I see a benefit by using ES?

[RobParker]

We've just moved to using a separate server for MySQL and this has finally let me rebuild my search index without killing our site.

We have apprx 2.5M posts and the MySQL server is a VPS with 1 CPU and 2GB of ram and is running smoothly.

Since rebuilding the search index, search results are very slow 5-10 seconds to return so I was considering ES but as the VPS only has 2GB of ram and half of this is set aside for the innodb buffer, I wonder whether ES would be any use...?

 

[Slavik]

You were on a mini cluster correct?

Can you just re-post the specs of each part of the cluster?

Depending on the setup.

1) Use your most powerful server to host the ES service on its own and use custom mapping to drastically cut down the size

or

2) Run the ES service across all the servers.

 

[RobParker]

The web server is a "cloud" server so has configurable resources which we can up/down as required.

We're currently setup with 2 cps and 2gb of ram (upped to 4gb of ram when we're busy). The MySQL server is 1 CPU and 2 gb of ram and typically a very low load.

After rebuilding the search index earlier our database has jumped from 2GB to 4GB...

 

[Slavik]

The web server is a "cloud" server so has configurable resources which we can up/down as required.

We're currently setup with 2 cps and 2gb of ram (upped to 4gb of ram when we're busy). The MySQL server is 1 CPU and 2 gb of ram and typically a very low load.

After rebuilding the search index earlier our database has jumped from 2GB to 4GB...

Well 2.5M posts on a mysql search is going to suck any day of the week.

If your mysql server is typically very low load then I would try setting it up on there and use the mapping that IGN has been using to reduce the size. Hopefully this should keep everything happy.

Unfortunately I have not done much with clustering ES so I couldn't advise you on that, however I believe Deebs has been experiementing with clustering so may be of more help if you wanted to go down that route.

 

[RobParker]

But what happens if I only assign 500MB to ES? Will it just fail or will it still be better than doing the mysql search?

Does it need to contain the entire search index or is it more like a cache where it only contains the latest searches and the old ones drop out when full?

 

[Slavik]

But what happens if I only assign 500MB to ES? Will it just fail or will it still be better than doing the mysql search?

Does it need to contain the entire search index or is it more like a cache where it only contains the latest searches and the old ones drop out when full?

It depends how hard your search is being hammered and whats being searched for. If you don't have enough memory allocated it will start showing not enough memory errors.