XF 1.5 Major problems with search in Xenforo compared to SMF...

[Overscan]

I've finally got my SMF forum migrated to Xenforo 1.5 attachment intact and permissions fixed (will upgrade to 2.1 tomorrow) and I am just doing basic functionality tests.

My forum is part discussion and part reference database. People regularly search the forum looking for information or old topics to which they can add new information. Search is fundamental to my forum's existence.

So, in SMF 2.0 version of my forum, I am looking for a topic I know exists on the design evolution of the F-16 fighter from 1965 - 1972.

In the SMF search box I type

Evolution F-16

The very first result I get is a link to the topic:

ADF to LWF: Evolution of the General Dynamics F-16

If I repeat the exact same search in Xenforo I get:

Search Results for Query: evolution f-16

1. The following words were not included in your search because they are too short, too long, or too common: f, 16

And basically it just searches for 'evolution' and my topic is buried deep in the results.

If by some miracle I remember the full and complete name of the topic and search for that:

Search Results for Query: ADF to LWF: Evolution of the General Dynamics F-16

1. The following words were not included in your search because they are too short, too long, or too common: ADF, to, LWF, of, the, F, 16

It gives me the actual topic I'm after in 8th position, due to throwing away most of the relevant keywords for being too short.

So I figure out two problems. First, I need to alter MySQL to allow 3 character searches, then update Xenforo to match. I can search for ADF or LWF now, though F-16 is still impossible as it searches for F and 16 separately.

Second, I need to put the search query in quotes. This now works a little better, but it still puts my desired topic notably lower down than SMF.

I check SMF forum search settings:

Relative search weight for number of matching messages within a topic: 20%
Relative search weight for age of last matching message: 0%
Relative search weight for topic length: 20%
Relative search weight for a matching subject: 40%
Relative search weight for a first message match: 20%
Relative search weight for a sticky topic: 0%

I was able, 13 years ago when I set up the SMF forum, to adjust the search weightings - remove age of message from weight, prioritize subject matches - and by and large, it's worked fine and similarly to Google - put in some keywords and find relevant content.

Xenforo doesn't seem to have any options similar so far. Is this a case where I'll need to buy the XenForo Enhanced Search addon? Even that seems to have very few 'knobs' to tweak as it were.

 

[Mike]

I don't know exactly how SMF implemented their search system. It sounds a little bit like it might be a custom system. This does give some potential for additional options, though performance is often an issue there.

In terms of the out of the box search, MySQL's full text search system is used. This has a global minimum word length that would require root access to change. It defaults to 4. There's some discussion of changing this here: https://dev.mysql.com/doc/refman/8.0/en/fulltext-fine-tuning.html#fulltext-word-length (Note that we use the MyISAM version.)

Note that out of the box, the results are date ordered. This primarily has to do with MySQL's limits regarding relevancy and boolean search (with MyISAM FT searching in particular).

In terms of the "f-16" examples, unfortunately that is generally controlled by MySQL's tokenization system (which splits the contents and search terms into separate words), making that into "f" and "16" which then get rejected for length limits. (MySQL also has a large stop word list which are words that will never get indexed.)

With the Enhanced Search add-on, we use Elasticsearch (which you would have to have the ability to install). This supports indexing 1 letter words if desired, relevancy-based results (using an internal algorithm, plus we boost for title matches; you can also control whether age is considered), and stemming options (so words like "cat" and "cats" are both found in results). This should help better results be returned (and more consistently).

 

[AndyB]

People regularly search the forum looking for information or old topics to which they can add new information. Search is fundamental to my forum's existence.

You might like this add-on:

https://xenforo.com/community/resources/advanced-search.3781/

 

[RobinHood]

You could also try this https://xenforo.com/community/resources/threadloom-search.5262/

 

[TPerry]

Or you could teach your users to use the pipe command between search term words.
Evolution|F-16 seems to work pretty well.

 

[Overscan]

Thanks guys, I'll give those suggestions a try. I'm not opposed to Elasticsearch, I've used that at work a bit. I think they key for my needs is ignoring ages of posts by default.

 

[Overscan]

So - I've bitten the bullet and purchased the Enhanced search add-on. It is doing a great job of indexing and search. I'm slightly annoyed I needed an paid addon to deliver essentially core functionality - but overall my Xenforo experience is going great.

So I'm pretty much down to theming to get my forum ready for migration day.