What is the most basic DO Droplet that would support XFES 2.2 features on a 1.4m forum

Chromaniac

Chromaniac

Well-known member
So, I purchased XFES few years ago because the hosting server I was going to use offered ES support as part of the package. I got scammed by that host (story posted elsewhere on this board) so I did not renew the add-on as I never ended up using it and didn't expect to use it in the future. That turned out to be a mistake.

I wouldn't have bothered now but the new features in 2.2 powered by XFES are things I am very interested in. My forum is low traffic hence easily survives on a basic 5 dollars DO Droplet. But it is also quite old and has a lot of **** posting which means it does have 1.4 m posts.

Now I am considering getting XFES when I renew my subscription. But I am concerned about the costing of the second droplet I would need to run Elastic Search. I was just wondering if anyone here can point out the server requirement for a droplet running Elasticsearch alone that would be indexing and serving the new 2.2 features on a low traffic forum with around 1.4m posts. Similar Threads I assume would be the most resource intensive feature I imagine. I am not sure if the content is cached or not... I do not remember anything about this on the HYS article. Even then, it is likely to hit the second droplet constantly.

I also wonder if XenForo devs would allow me to test out XFES on launch (I would of course pay for it) and if it turns out to be an expensive project... I can apply for a refund? Maybe @Brogan can provide a quick response on this. Though, I do think that XFES features are the ones I am most excited about in 2.2 (apart from PWA) so I would very likely manage with whatever it ends up costing me on a monthly basis...

Thanks all.
 
Sort by date Sort by votes
rdn said:
The cheapest $5 should work fine for you.

Budget-Friendly Cloud Server Pricing | DigitalOcean

Need a powerful cloud server without the hefty price tag? DigitalOcean has got you covered. Enjoy exceptional performance and flexibility at unbeatable prices.
www.digitalocean.com www.digitalocean.com
Click to expand...
whoa. Are you sure? From what I have read googling around... java alone consumes a lot of ram. Anyone here has success running an Elasticsearch install on a 1CPU/1GB RAM VPS?

On related note... What is better? Running forum and ES on different droplets? Or just get a more powerful droplet and run both on same?

Also... Which of the three options is best for Elastic Search... 1CPU/3GB, 2CPU/2GB, 3CPU/1GB. In case, a 5 or 10 dollar droplet does not work well eventually? Thanks!
 
Upvote 0 Downvote
I have considered them in the past, but I am quite happy with Singapore based servers on digital ocean. All of my traffic is based in India and there is not much performance hit with hosting in SG. Europe is a bit slower. I assume using Elasticsearch on a far off location would add lag unless I move the forum to the same data center as well?
 
Upvote 0 Downvote
Right. Based on your input, I think I should be able to make do with 10/15 dollar droplet in the worst case scenario. Was just wondering what is more important for an Elasticsearch based install... RAM or CPU. DO has three options for USD 15 droplet (1CPU/3GB, 2CPU/2GB, 3CPU/1GB). Which one should I focus on if need arises? Thanks!
 
Upvote 0 Downvote
I'm not sure how complicated it is to set up - fair to say a bunch of AWS services can be somewhat involved to set up - but Amazon AWS does have an Elasticsearch hosting and they have what looks like a reasonable free tier.

Not looked into it a bunch but it might be a good way to use ES without committing to any cost.
 
Upvote 0 Downvote
Yes. Aws Singapore is the backup plan if I fail to get it working on DO. Not sure how much lag it would introduce even with both datacenters in same region. Would report back if I end up using them. Thanks!
 
Upvote 0 Downvote
Update time! I got Elasticsearch working finally with a lot of help from a friend. It managed to install just fine on the USD 5 droplet but it kept on crashing during indexing. So I upgraded it to the 2GB RAM edition and it has been working pretty well since then. I love how I can search for tiny things now. Had never noticed it here because I had conditioned myself to use longer search terms! Excellent stuff.
 
Upvote 0 Downvote
Question. ES has been working great on my 2.1.x install. But every couple of days out of the blue search would stop working and server error would indicate that the search index does not exist. Rebuilding fixes the problem but since server errors are not sent as email updates, I do not discover the issue unless someone reports or I perform a search myself. Here is an error from few minutes ago...

Code: 
XFES\Elasticsearch\RequestException: Elasticsearch error: no such index [INDEXNAME] src/addons/XFES/Elasticsearch/Api.php:409
Generated by: MYNAME Aug 25, 2020 at 9:11 PM
Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(330): XFES\Elasticsearch\Api->request()
#1 src/addons/XFES/Elasticsearch/Api.php(234): XFES\Elasticsearch\Api->requestFromIndex()
#2 src/addons/SV/SearchImprovements/XFES/Elasticsearch/Api.php(21): XFES\Elasticsearch\Api->search()
#3 src/addons/XFES/Search/Source/Elasticsearch.php(153): SV\SearchImprovements\XFES\Elasticsearch\Api->search()
#4 src/XF/Search/Search.php(175): XFES\Search\Source\Elasticsearch->search()
#5 src/XF/Repository/Search.php(34): XF\Search\Search->search()
#6 src/XF/Pub/Controller/Search.php(402): XF\Repository\Search->runSearch()
#7 src/XF/Pub/Controller/Search.php(113): XF\Pub\Controller\Search->runSearch()
#8 src/addons/SV/SearchImprovements/XF/Pub/Controller/Search.php(58): XF\Pub\Controller\Search->actionSearch()
#9 src/XF/Mvc/Dispatcher.php(350): SV\SearchImprovements\XF\Pub\Controller\Search->actionSearch()
#10 src/XF/Mvc/Dispatcher.php(257): XF\Mvc\Dispatcher->dispatchClass()
#11 src/XF/Mvc/Dispatcher.php(113): XF\Mvc\Dispatcher->dispatchFromMatch()
#12 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop()
#13 src/XF/App.php(2190): XF\Mvc\Dispatcher->run()
#14 src/XF.php(391): XF\App->run()
#15 index.php(20): XF::runApp()
#16 {main}

ES is running on an independent droplet which has enough free RAM. It also has enough free disk space. I understand there are lots of threads on this already. But was just wondering if there is something wrong with my configuration that can fix this.
 
Upvote 0 Downvote
Next time it happens, check on the server whether the index actually exists.

If it doesn't then further investigation would be needed to determine why it's being deleted.
 
Upvote 0 Downvote
I have a similarly sized site, 1.5 mil posts, but not as active as it once was in the hayday.
i run a 4gb / 2 core vps with ES on the same machine.


Documents 1,460,892 (263.1 MB)
Index updates 70
Searches 4,795 (1 milliseconds average)
Allocated memory 956.5 KB
 
Upvote 0 Downvote
shouldn't this value be larger than documents?

Index updates 70

for me it's

Code: 
Documents    1,391,184 (770.9 MB)
Index updates    1,518,628
Searches    17 (11 milliseconds average)
Allocated memory    68.8 KB

my allocated memory also is very low compared to yours. is this connected to values configured in ES settings? :confused:
 
Upvote 0 Downvote
You must log in or register to reply here.
Back
Top Bottom