XF 2.1 Improving searching - best practices and add-ons

salata

Member
Hi all,
I inherited small community xen based forum (DB size is 1GB, approx 100 active users) and review current status. The obvious improvement is search engine. Forum is hosted and current amount of RAM is 6GB. My concerns is ElasticSearch and how demanding on RAM it is. Also licensing model of ES may be an issue. Are there any options that not require ES to improve search? Also what is the best way (with or without ES) to improve search.

The last but not least, what is necessary to have 'Has this question already been asked?' feature appear when you start new topic?

best,
S
 
Hello,

Are there any options that not require ES to improve search? Also what is the best way (with or without ES) to improve search.

The last but not least, what is necessary to have 'Has this question already been asked?' feature appear when you start new topic?
 
Down to basic nut-cutting.. you want good search, use ES. You want standard piss-ant mySQL sub-standard search, use mySQL.
There is a reason that search engines like Sphinx & ElasticSearch are used.
 
Not everyone can afford a dedicated server, a VPS or to subscribe to XF Cloud, so some have to stick to SQL searches and try to find solutions to improve them. I would love to use XFES on my forum but that time hasn't arrived yet.
 
Not everyone can afford a dedicated server, a VPS or to subscribe to XF Cloud, so some have to stick to SQL searches and try to find solutions to improve them. I would love to use XFES on my forum but that time hasn't arrived yet.
And the issue is... there are NO good outside solutions. You will either be dependent upon a an outside service that you have to pay for, you have to have ES capability (which also entails additional costs), or you have to live with what mySQL is capable of.
You don't have to have a "dedicated server" for ES.. you can simply have another VPS instance running ES or subscribe to a service that offers ES connectivity.
 
Hey ! ES standard plan starts at $95/month... currently my server costs me 80€/year...
Granted.. it may be expensive.. but wanting "special" things often are. Hell, iI'm "wanting" a new mount for my scopes...and $10.5k is a little out of my budget... but if I "required" it, I'd find a legal way to obtain it.
 
I'll look into @MattW's offer, my contract ends in February, it's not that far away.
If nothing else, I can promise you that unless @MattW has drastically changed recently, you will be MORE than satisfied with his service. He's honestly the ONLY other person that I know that I would personally trust with hosting my site.
 
For me having better search is to have accuracy in the first place. First I will purchase search improvements module. I am talking to my hosting if they permit installation of ES, but I don't know what would be impact on the resources. Those wo are using ES, can you tell how much RAM it consumes? Other option may be VPS, but I'd rather have it running locally. I still have 4GB of RAM free on my instance but not sure if this may be enough.
 
If nothing else, I can promise you that unless @MattW has drastically changed recently, you will be MORE than satisfied with his service. He's honestly the ONLY other person that I know that I would personally trust with hosting my site.
Nope, nothing has changed, other than business is continuing to grow :)

Those wo are using ES, can you tell how much RAM it consumes?
It depends on how much you configure it to use. 2.6million post forum:
1665734586957.webp
 
I've got a paid add-on ElasticSearch Essentials:

It has a number of search options which do help improve search accuracy. "Exact text" search while having stemming is a powerful feature, and autocomplete is very useful for some people.

XenForo Enhanced Search has a similar thread implementation, no point paying for just that feature.

And the issue is... there are NO good outside solutions. You will either be dependent upon a an outside service that you have to pay for, you have to have ES capability (which also entails additional costs), or you have to live with what mySQL is capable of.
ElasticSearch can use less memory than MySQL for it's full-text search while having vastly better performance. and while using less diskspace.

Also licensing model of ES may be an issue.
Are you selling a ElasticSearch instance directly to customers? If not, then it doesn't matter.
 
ElasticSearch can use less memory than MySQL for it's full-text search while having vastly better performance. and while using less diskspace.
I'm WELL aware of that.. but the simple fact remains, can you show me ANY shared hosting provider that will allow you to install ES engine on their platform?
On a marginal (2GB) VPS that is somewhat busy, ANY additional load detracts from the performance, and since mySQL is already loaded and the impact, depending on ones configuration, is not as drastic as adding another service utilizing ADDITIONAL resources over what an already running instance of mySQL has can be telling. Or are you trying to say that ES engine itself has NO additional overhead, and simply will use less than what mySQL search itself uses for the TOTAL instance of ES? I think you and I both know the answer to that question.
To utilize ES efficiently you need a MINIMUM of 1GB over what you normally are using.
It is recommended to have at LEAST 4GB of RAM last time I checked. You and I BOTH know that you need additional resources that shared hosting, and most base VPS's are not capable of providing for ES in their initial configuration (specifically a VPS). So one will have to engage an outside provider for ES, increase their VPS specs or get a second VPS and utilize it.
 
Last edited:
Top Bottom