ES 2.2 Enhanced Search 2.2 - Similar Threads

In the beginning, XenForo Enhanced Search (XFES) was intended to be a simple drop-in replacement for MySQL full text search, primarily aimed at large forums whose operation would potentially suffer when attempting to search very large database tables.

As a drop-in, XFES only made use of the enhanced search index and speed for the formal search system, but with version 2.2 we are starting to leverage more of the power afforded by the Elasticsearch back-end.

Similar threads​

A useful tool for surfacing content on your forum is a list of Similar or related threads, which can be displayed along with a primary thread. Visitors who have landed on that page from a search engine will often look for related material to continue their reading, and Similar threads offer a great way to provide that in an automated fashion.

One limitation of the usefulness of Similar threads is a determination of just how similar or related the results actually are. This is where XFES comes into its own, as the Elasticsearch back-end provides a specific ability to search for more like this using the current thread as a search parameter. Given a decent amount of content from which to pluck similar results, this provides an excellent list of threads in which the viewer of the current thread may be interested.

Similar threads widget​

We display our similar threads results in a widget, and by default we place it below the quick reply area on thread view pages, but as it is a widget you can choose to place it wherever you like on the page.

You could even choose to place multiple instances of the widget on the page, and make use of the power of the widget system to control how each instance of the widget behaves, setting parameters such as the number of thread results visible, the style of display, the source forums searched and the time period within which to search.

Controlling the index​

Of course, it's possible to fine-tune the way in which results are found, and we provide options to positively-weight results based on being in the same forum as the source / original thread, and based on how new the related content may be.

But wait, there's more...

The extreme speed and efficiency of XFES allows us to offer another, related tool to our arsenal: similar thread suggestion at thread composition time.

Suggest while composing​

Everyone hates being told that a question they've asked has already been dealt with, or that the topic about which they have posted has already been covered.

XFES 2.2 aims to minimise the occurrence of this, by suggesting related threads that may contain content similar to that which the user is about to post, while they are composing. XFES is fast enough that sensible suggestions can be made in real-time while the user is typing the title of their new thread, which may prompt them to visit one of the suggested existing threads rather than posting a potential duplicate.

---

​

---

The thread suggestion system can be a real game-changer for unwanted duplicate threads.

The system can be turned on and off through the control panel, along with options for how many similar threads to suggest.

It's also possible to prevent the suggestion system from appearing at all on a per-forum basis. For example, we would never want to be pestered with suggestions for similar threads when composing a new release announcement, or a Have You Seen thread.

In conclusion​

We think you'll agree that the addition of Similar threads functionality to XenForo Enhanced Search adds a compelling new ability to an already very powerful system.

By way of a reminder, XFES requires that your server has Elasticsearch 2.0 or newer, and can be purchased here.

 

[Chris D]

How, though?

If you put a list of similar threads in, say, a media gallery category, what are these threads similar to?

 

[Alpha1]

Good question. This leads to the question:
Are thread content and tags used for the result of similar threads?

If yes, then in the case of a media gallery image, then it would search for similar threads based upon:

• title
• textual content
• tags

In the case of categories, it makes no sense to post a similar threads widget there. Thats the same for forum. But it would make sense to have it for: images, resources, or for third party addon content like events, articles, blog entries, showcase entries, wiki page, etc.

 

[Kirby]

How, though?

If you put a list of similar threads in, say, a media gallery category, what are these threads similar to?

A media gallery category probably would not make much sense, but a media item or resource would - it does have an author, title, body text and category it does belong to just like a thread.

 

[Chris D]

Sounds like a suggestion to me.

 

[Kirby]

Yep
Do you want me to "officially" post it as one?

 

[Chris D]

Always. We’ll never find it again otherwise.

 

[beerForo]

Another cool feature!

 

[Kevin]

Just one question.... Are matching results shown from the 'Search' drop-down box also?

 

[Alpha1]

Always. We’ll never find it again otherwise.

Already here:

Similar Content Widgets (for any content type)

Content Discovery is a major issue with XenForo when you have addons. When a user is browsing a forum or thread about a specific content, then the user is most likely interested in the very same topic in other content types. But this related content is often hidden in a category of an addon...

xenforo.com

It also asks for a 'similar resources' and 'similar media' widgets

 

[Wildcat Media]

This is a great feature and it can probably trim another add-on from my collection.

The only other feature I've wanted for years is phonetic searching (Soundex, metaphone, etc.), which is available in Elasticsearch through a plugin. I think that would kick up the search results quite a bit--take a look around the Internet at how many words are mispeld misspelled and you can see why it's helpful. Even sometimes when I have to guess on a proper name, Google can usually figure out what I mean--a search engine looking for an exact match would never retrieve that result for me.

I feel it would take search results to the next level. We do have members complain about the "horrible forum search engine" and aside from educating them on how to search smarter, we can't be their spell-checker.

I made a suggestion for this a couple of years ago but surprisingly there was no interest in it.

 

[beerForo]

This works with the new thread types correct? Similar questions, articles... and in a mixed content forum, you can pull similar articles only, similar questions only, etc? And the language changes accordingly?

 

[sbj]

Nice update. Though I am suprised not see the most useful thing of a search system: auto-suggest.

 

[Alpha1]

IF I understand you correctly you are not asking for this auto-suggest feature:

https://xenforo.com/community/data/video/230/230448-e7568ea64c998caa10dfe7fea8b3d11d.mp4

But instead for suggestions for the most popular queries.

 

[sbj]

Not entirely correct.

"Suggest while composing" is a nice feature but how about a "suggest while searching" instead?

Xon calls it "search bar auto complete".

Just like on Google. When I search something, the search bar gives me results while I am typing.

Now, of course the system can be tweaked, if the auto complete is based on most popular queries, or relevance, or by newest date, and so on. That is independant from where the suggestion happens.

I am just suprised that we get the suggestions on thread creations, which is a secondary place for searches and not on the primary interface on the search bar itself.

It is called ElasticSearch and not ElasticThreadCreation . One would think the search bar gets it first before anything else .

 

[Chris D]

Although this solves a more significant and prevalent problem that I’m sure all forum admins acknowledge - that many users create threads when an existing thread is the appropriate place to post.

This should alleviate that issue.

I know you’re joking but to be clear the method of searching for similar threads uses the search system so it fits entirely appropriately under the remit of Elasticsearch without us having to worry that the title of the add-on is no longer semantically correct

 

[sbj]

Yeah, the title of the addon is fine as it is of course .

And I totally agree that the current new feature helps solving an important problem. Nothing wrong with that.

But I don't think it is the more significant prevalent problem.

I am a member of this community since 2013 and if I counted correctly, I have in total 147 threads created (compared to my 2433 messages).

And what do you think over this time, how often did I use the search system? Less than 147 times?

So this new feature could have helped me in 147 instances probably. But I can guarantee you I used the search system at least 10x times more than that and an auto suggest feature would have saved me a lot of time. It probably could have stopped me in creating a thread in first place, before I come across the "suggest while composing".

I believe that is the more significant advancement.

 

[frm]

I never had ES, so forgive me if this sounds like a silly question...

Do we need to install ES for all functionality, even similar threads? Or does the similar threads widget work independently and not reliant on ES (besides the add on) and auto-complete works only with ES installed?

 

[KSA]

Is there a field where we can omit certain phrases/words not to be included when showing results of similar threads e.g and, or, when, what, on, it,?

Thank you for answering my question.

 

[Kirby]

Do we need to install ES for all functionality, even similar threads?

You do need access to an ES instance in order to use XFES. The Similar Threads functionality is part of XFES and thus does require ES as far as I understood @Chris D.

 

[BIG LLC]

Can we limit the date range? So a thread from two years ago would probably not be relevant, but one from two days ago almost certainly would.