Search breaks when short/common words used?

[Fullmental]

Affected version

2.2

I recently tried searching for a thread on our forum by exact thread name that I knew existed. I tried this using the "search threads only" criteria, but searching for the thread using the keywords returned zero results. Attempting to search for the same keywords on a standard search with no options returned the thread near the bottom of a long, long list of results, and included this message, but only once:



After searching once, this message ceased to appear in subsequent searches, and the standard keyword search stopped showing ANY results. Very odd!

Interestingly enough this message does not appear even once on a search where "search threads only" are checked, and instead of ignoring the keyword the inclusion of such words seems to just break the search results entirely. Searching for the same thread title with "on" excluded returns the thread, so I know the thread is in the search index.


Still, I did some checking to make sure everything was as expected. The search indexer is enabled at the node level, and I just tried a complete rebuild of the thread search index both with and without deleting the existing cache, with no change:



There are no errors in the ACP, and no indications as to why a user might find zero results.

The expected behavior would be, if the search engine cannot use these common words, they would get excluded just like in a search for all content. At the very least, a message should always be displayed to the user to try removing common or short words, and provide a list so they can manually adjust their search criteria.

Note: we do NOT have enhanced search. It is part of our license, but it is not enabled on our production site yet and the addon has not been installed. The issue appears to be fixed when using enhanced search on a test site.

 

[Xon]

The built-in MySQL full-text search is quite limited. If you have an XFES licence, I really do recommend using it.

ElasticSearch is a vastly more flexible solution, and can be extended with custom search logic which gives real usable improvements to search.

I've got a couple of (paid) add-on which add various nice-to-have features powered by ElasticSearch's more robust and complete feature-set.

 

[Fullmental]

The built-in MySQL full-text search is quite limited. If you have an XFES licence, I really do recommend using it.

ElasticSearch is a vastly more flexible solution, and can be extended with custom search logic which gives real usable improvements to search.

I've got a couple of (paid) add-on which add various nice-to-have features powered by ElasticSearch's more robust and complete feature-set.

Yes, it does make sense that the search would be limited and we are working on testing an ElasticSearch/XFES/Addon-based configuration to expand search compatibility, but it is not ready yet.

Still, I would expect an appropriate message to appear consistently so the user knows there's something affecting results based on the keywords they searched with, rather than returning an empty result as if what they're looking for doesn't exist.

 

[cdub]

I recently tried searching for a thread on our forum by exact thread name that I knew existed. I tried this using the "search threads only" criteria, but searching for the thread using the keywords returned zero results. Attempting to search for the same keywords on a standard search with no options returned the thread near the bottom of a long, long list of results, and included this message, but only once:

View attachment 272696

After searching once, this message ceased to appear in subsequent searches, and the standard keyword search stopped showing ANY results. Very odd!

Interestingly enough this message does not appear even once on a search where "search threads only" are checked, and instead of ignoring the keyword the inclusion of such words seems to just break the search results entirely. Searching for the same thread title with "on" excluded returns the thread, so I know the thread is in the search index.


Still, I did some checking to make sure everything was as expected. The search indexer is enabled at the node level, and I just tried a complete rebuild of the thread search index both with and without deleting the existing cache, with no change:

View attachment 272694

There are no errors in the ACP, and no indications as to why a user might find zero results.

The expected behavior would be, if the search engine cannot use these common words, they would get excluded just like in a search for all content. At the very least, a message should always be displayed to the user to try removing common or short words, and provide a list so they can manually adjust their search criteria.

Note: we do NOT have enhanced search. It is part of our license, but it is not enabled on our production site yet and the addon has not been installed. The issue appears to be fixed when using enhanced search on a test site.

Yes I had this exact thing happen. I searched for "I alone" and got the error that it's too short and now searching for "alone" gets nothing when I know posts have it. Granted this is with search everything. Search threads and search IMS items returns proper results.

I'd forgotten how bad regular XF search is. This is for a new site and I've been using enhanced search on my regular site since I switched in 2014.

This seems more than just bad search. It's removing a word from the index. I rebuilt search index and nothing.

I too will be getting enhanced search soon.

 

[Paul B]

I'd forgotten how bad regular XF search is

It's standard MySQL search, not an XF product.

 

[cdub]

It's standard MySQL search, not an XF product.

Yes but it's a feature within an XF product and there's no distinction to my users. I'll be buying enhanced search later today.

 

[TPerry]

Yes but it's a feature within an XF product and there's no distinction to my users.

It's a feature with just about any product that uses default mySQL for the search engine. SMF is not any better. myBB is not any better. IPS is not any better. WoltLab is not any better.
That's why I broke down and got ES for my site.... not because it's gigantic and has lots of content, but because I wanted users to be able to do searches on stuff like M1 or M82, which standard mySQL is not set up for and you have to jump through hoops to get mySQL to do 2 letter searches, and even THEN it still is horrible.

 

[AndyB]

You might like this add-on:

https://xenforo.com/community/resources/advanced-search.5602/