[Solved] Searching HandBall Versus Hand Ball (space)

[John917]

I have a big issue when I search with enhanced search, it can't detect HandBall results whenever I search with a space in between.

So I want to find "HandBall."

I search "Hand Ball" with a space instead. This search query results with nothing when I want it to show HandBall.

Any idea how to fix this?

 

[Paul B]

Enable stemming.

 

[John917]

Hi Brogan,
It's wierd since I'm pretty sure I have that enabled.
I'm actually searching something more like a word and a number. So if I want results: android123 but search "android 123" it won't work. I'm gonna try searching this now on this forum to see if stemming is applied with numbers.
*Yea it works here. What am I doing wrong...

 

[Xon]

Changing index configuration options requires a full reindex, and depending it may require truncating the data which actually deletes the index and then recreates it.

Sadly this can be different from just updating the index because ElasticSearch is a little janky at times.

 

[John917]

Yea, are you referring to the option to delete search index when recreating it in ACP?

Or im assuming full reindex is done on the elastic search server?

Please note I'm running ES on a remote server. Things seem to be running fine and can't find any errors. Just the darn stemming feature isn't working.

Can someone at Xenforo look at my server configs? So bummed I have paid for a great product but seems to be a bug.

By the way, stemming does work with numbers so its gotta be something else (attached)

Changing index configuration options requires a full reindex, and depending it may require truncating the data which actually deletes the index and then recreates it.

Sadly this can be different from just updating the index because ElasticSearch is a little janky at times.

 

[mjda]

Yea, are you referring to the option to delete search index when recreating it in ACP?

He's talking about rebuilding your search index, in the ACP, as seen here:

By the way, stemming does work with numbers so its gotta be something else (attached)

From what I'm seeing, your search for "android 123" didn't return a result for "android123". It returned results for "android" and "123" separately. Notice that "android123" isn't all highlighted (or bold) in your search result.

 

[John917]

He's talking about rebuilding your search index, in the ACP, as seen here:

View attachment 276634




From what I'm seeing, your search for "android 123" didn't return a result for "android123". It returned results for "android" and "123" separately. Notice that "android123" isn't all highlighted (or bold) in your search result.

Yes that's what I meant for both your posts.

That stemming is what I want, but doesn't work on my board.

Wish there was a way to properly troubleshoot this. Maybe it's with the addon however since no error logs as of yet.

Which log files do you think would help?

 

[mjda]

Yes that's what I meant for both your posts.

That stemming is what I want, but doesn't work on my board.

Wish there was a way to properly troubleshoot this. Maybe it's with the addon however since no error logs as of yet.

Which log files do you think would help?

I don't think your issue has anything to do with the stemming option. I have the stemming option enabled on my forum and something similar happens with my search results too. I actually hadn't ever noticed it before today, but I just looked and realized my issue is the same as yours.

The search results for "foot ball" and "foot balls" are the same, so I know the stemming option is working. However, the search results for "football" are different. It's also worth noting that, on my site, no results for "football" are returned if you search for "foot ball".

Also, to @Brogan's reply, I don't think the search is actually finding the word "handball" in the search for "hand ball" here. It's just finding the words "hand" and "ball" separately within the thread and returning that. Of course, I don't know that without testing it. I'm just assuming.

EDIT: I just created a thread in the test forum called "handball". You cannot find it in the search results if you search for "hand ball". You must search for "handball" to get it to show up.

 

[John917]

I don't think your issue has anything to do with the stemming option. I have the stemming option enabled on my forum and something similar happens with my search results too. I actually hadn't ever noticed it before today, but I just looked and realized my issue is the same as yours.

The search results for "foot ball" and "foot balls" are the same, so I know the stemming option is working. However, the search results for "football" are different. It's also worth noting that, on my site, no results for "football" are returned if you search for "foot ball".

Also, to @Brogan's reply, I don't think the search is actually finding the word "handball" in the search for "hand ball" here. It's just finding the words "hand" and "ball" separately within the thread and returning that.

Hey! Thanks for the help. I think Brogan and I are on the same page with you for stemming. If you look at my screenshots, we're referring to the same thing.

It's also worth noting that, on my site, no results for "football" are returned if you search for "foot ball".

That's a bug or something is wrong somewhere. I'm having the same issue. If you search "hand ball" it stems and returns results for handball. Like you said, it stems out the hand in handball.

 

[mjda]

That's a bug or something is wrong somewhere. I'm having the same issue. If you search "hand ball" it stems and returns results for handball. Like you said, it stems out the hand in handball.

I'm not sure I'd say it's a bug. I think things are working as intended by the developers, just maybe not as expected by some users. Of course only the developers could say for sure what their expectation would be in this case.

 

[Kirby]

IMHO, stemming doesn't really help here as hand ball and handball don't have a common stem.

I've created a little test thread that has two posts (one with handball and another one with hand ball) to demonstrate this.

Searching for hand ball


Searching for handball


As can be seen from the sreenshots, only one of those posts is returned in either search.

The screenshot posted by @Brogan might lead to the assumption that handball was matched in the first post (as hand is highlighted as part of handball), but this actually isn't the case (only hand ball was matched in title and message):
Highlighting is done in XenForo (with a rather simple approach) while matching is done in ElasticSearch; to get accurate highlighting it would be ncecessary to also do highlighting in ElasticSearch.

Design issue - Highlighting does not work reliably

If a search term is a stopword or appears as start of a word, it does get highlighted on the results page even though it was not being searched for. Steps to reprodue Configure Enhanced Search to use built-in stopwords for German Add content with title "Robust, leicht, top, zuverlässig - mit...

xenforo.com

The real fix for this usecase (apart from the highlighting) would be support for synonyms and decompounder as handball is basically a compound word made of hand and ball

 

[John917]

IMHO, stemming doesn't really help here as hand ball and handball don't have a common stem.

I've created a little test thread that has two posts (one with handball and another one with hand ball) to demonstrate this.

Searching for hand ball
View attachment 276648

Searching for handball
View attachment 276647

As can be seen from the sreenshots, only one of those posts is returned in either search.

The screenshot posted by @Brogan might lead to the assumption that handball was matched in the first post (as hand is highlighted as part of handball), but this actually isn't the case (only hand ball was matched in title and message):
Highlighting is done in XenForo (with a rather simple approach) while matching is done in ElasticSearch; to get accurate highlighting it would be ncecessary to also do highlighting in ElasticSearch.

Design issue - Highlighting does not work reliably

If a search term is a stopword or appears as start of a word, it does get highlighted on the results page even though it was not being searched for. Steps to reprodue Configure Enhanced Search to use built-in stopwords for German Add content with title "Robust, leicht, top, zuverlässig - mit...

xenforo.com

The real fix for this usecase (apart from the highlighting) would be support for synonyms and decompounder as handball is basically a compound word made of hand and ball

I think if I read this right, in simple terms.

Search: hand ball
Desired results: handball
Works or not: yes, with stemming.

Search: handball
Desired results: hand ball
Works or not: no since this isn't how stemming works.

Either way, going back to my original post, the normal stemming feature is enough for me but it's not working as it should on my server. Support is looking into it right now. Thank you Xenforo.

 

[mjda]

Search: hand ball
Desired results: handball
Works or not: yes, with stemming.

No, this won't work, even with stemming. Look at the search results @Kirby posted. If you search for "hand ball", you'll only get search results for "hand" and "ball". You won't get results for "handball", whether stemming is enabled or not.

 

[John917]

No, this won't work, even with stemming. Look at the search results @Kirby posted. If you search for "hand ball", you'll only get search results for "hand" and "ball". You won't get results for "handball", whether stemming is enabled or not.

Ah well I guess if you're talking about the technical side of things. But in practicality, it does give results for handball "indirectly" by pushing results for any words that have a prefix of "hand."

Which I was trying to get working since it's not working for me for some reason. Works well on Xenforo's site.

 

[mjda]

Ah well I guess if you're talking about the technical side of things. But in practicality, it does give results for handball "indirectly" by pushing results for any words that have a prefix of "hand."

No, that's not what is happening either. XenForo is only highlighting the word hand as a prefix to the word "handball". That's not why it's coming up in the search results, though. It's showing up because the words "hand" and/or "ball" are elsewhere in the post. Notice how neither his test thread or my test thread are showing up in the search results for "hand ball" but are both showing up in the search results for "handball". That's because they only use the word "handball" and not "hand" and/or "ball" separately.

 

[Kirby]

But in practicality, it does give results for handball "indirectly" by pushing results for any words that have a prefix of "hand."

No. Searching for hand ball doesn't match  handball and searching for handball doesn"t match hand ball - no matter if stemming is enabled or not.

Works well on Xenforo's site.

It doesn't

 

[John917]

No, that's not what is happening either. XenForo is only highlighting the word hand as a prefix to the word "handball". That's not why it's coming up in the search results, though. It's showing up because the words "hand" and/or "ball" are elsewhere in the post. Notice how neither his test thread or my test thread are showing up in the search results for "hand ball" but are both showing up in the search results for "handball". That's because they only use the word "handball" and not "hand" and/or "ball" separately.

Thank you for understanding my post. Gonna post a test post to test this out.

 

[John917]

gunbound

 

[John917]

filler145

 

[John917]

Filler145 filler