*Patiently waits for SSH to connect*
New Search functionality
- Thread starter Edrondol
- Start date
*Patiently waits for SSH to connect*
Deebs
Well-known member
As if, you want to play!
Forsaken
Well-known member
This add-on is for big boards, where search is one of the main consumer of resources on the server. While you might need it, the 50+ big boards that are currently on XenForo, and the ones planning to convert over obviously do.
lazy llama
Well-known member
Hmm... from those sizes I fear this is going to be a struggle for us as we're currently memory constrained.
Sphinx's searchd currently has a size of 435mb of which just 1.4mb is resident most of the time (it jumps to a couple of hundred meg when processing deltas). That handles our 10M+ posts very well.
Losing lots of RAM to a JVM and ElasticSearch isn't going to help InnoDB performance.
I'm hoping it'll work for us but it does look very memory heavy from what's been posted so far.
I'd be interested to see what the minimum required specs are going to be for the new Enhanced Search.
Sphinx's searchd currently has a size of 435mb of which just 1.4mb is resident most of the time (it jumps to a couple of hundred meg when processing deltas). That handles our 10M+ posts very well.
Losing lots of RAM to a JVM and ElasticSearch isn't going to help InnoDB performance.
I'm hoping it'll work for us but it does look very memory heavy from what's been posted so far.
I'd be interested to see what the minimum required specs are going to be for the new Enhanced Search.
Deebs
Well-known member
I am lucky in that the server I placed ES on has 20gig of RAM, 8gig of which is allocated to the slave MySQL instance, the rest is spare. I guess it could run with less memory but you would then push the i/o back down onto the disk subsystem. I hazard a guess as that is what your Sphinx installation is doing, your ondisk index size is going to be bigger than 435mb?
When I first started testing I was using ES with 512mb of RAM and it worked just fine. Didn't check the disk i/o. If I get time later this week I will have a play on my spare ES node with a restricted amount of RAM.
digitalpoint
Well-known member
Agreed.
For my current site, I use a Sphinx-based "big board" search that I had to build myself. And for me, $10/year is a very small price to pay to not have to develop/maintain a big-board search.
Search queries tend to be the nastiest queries your database server sees, so even if you don't need it as a big board, you can see benefits. From our Sphinx stats, we see a search roughly every 9 seconds...
An *average* search time of 0.003, even for the worst queries (and not needing to hit the DB server) is huge.
Now if we could just get master/slave DB support in XF...
lazy llama
Well-known member
Yeah, on-disk index is 4.3GB but we're not overly blessed on disk performance (everything is on a pair of Velociraptors in RAID1) and search is still blindingly fast with Sphinx.
I'll almost certainly buy it anyway to try it out. Even if it won't work for us straight off, one day we might get better hardware
For reference, without trying to do much tuning or be overly scientific, we did see ES running searches faster than Sphinx on a 5 million post board. Take that with a grain of salt though, as the tests were more for curiosity. Additionally, you don't get into managing delta indexes and merging like with Sphinx, which we have seen to potentially be a reasonable disk hit (that all gets managed for you and things are available instantly).
lazy llama
Well-known member
Instant updates will be nice, and faster searches for non-keyword, large-result-set searches would be excellent.
We currently build Sphinx deltas every 5 minutes, and run a full re-index once per day, so don't actually merge deltas (we're fortunate to have a relatively localised community so there are quiet times).
Anyway, I'm really glad this is (almost) here, as we've been holding off upgrading to 1.1.x pending a 1.1 compatible search solution.
I look forward to giving it (and our server) a good thrashing
We currently build Sphinx deltas every 5 minutes, and run a full re-index once per day, so don't actually merge deltas (we're fortunate to have a relatively localised community so there are quiet times).
Anyway, I'm really glad this is (almost) here, as we've been holding off upgrading to 1.1.x pending a 1.1 compatible search solution.
I look forward to giving it (and our server) a good thrashing
Digital Doctor
Well-known member
Agreed.
Price seems very low for the large boards that would need this.
Heres mine.
I increased the min and max mem limits of ES to 512 / 4096 respectively and it hasnt done anything in terms of usage, my total mem usage (including elasticsearch, java, mysql and httpd etc etc etc) is still floating around 1.5gb out of 8gb
In short, no.
Well, other than the stopwords list is smaller and there is no minimum word length.
For instance, you can search for the letter "b": http://xenforo.com/community/search/1729949/?q=b&t=post&o=date
For instance, you can search for the letter "b": http://xenforo.com/community/search/1729949/?q=b&t=post&o=date
2 Edge sword though.
A small site is most likely on shared / vps hosting and not capable of installing the elasticsearch service... unless you have an extremely flexible host.
True but I think it should be made more than clear this solution is for dedicated server / high end VPS users only, otherwise people may end up buying something they can never use.
Having said that, if you are technically capable of using the new search on a small board, go for it
Which reminds me, I may install elasticsearch on my customer dedicated server and see how it handles running several small Xenforo forums if they make the purchases
Similar threads
- Solved
