Implemented Enhanced Search - Similar Thread Caching

[nrep]

For larger forums with millions of threads, it would be very helpful to have similar thread caching for results (i.e. cache similar threads for 30 days) - ideally using a field in the xf_thread database, so that it works for long term caching. There are millions of threads which are old and still useful, but using ES to search for similar threads each time the thread is viewed would add a large overhead (especially when lots of different old threads are viewed).

 

[Mike]

This is what it does -- there's a table xf_es_thread_similar that holds this information. The results are cached for 14 days by default, though the flag to refresh this only happens when threads are viewed (so it doesn't end up refreshing every thread every 14 days).

Cache refreshes are done in the background, though the first trigger is done in the foreground. This is primarily so that when a new thread is posted we immediately have similar threads displayed.

 

[nrep]

Wonderful, thanks Mike.

 

[Chromaniac]

Had a question about 14 day caching. What happens if a thread included in the cache is deleted in that 14 day period? (Plus can this be configured? Tbh 14 days seems like a bit much. Would find out soon though!).

I am using Andy's add-on on 2.1 and it does not use caching. Every page load is hitting XFES I guess. Based on the fast increasing number in the XFES control panel.

 

[Paul B]

What happens if a thread included in the cache is deleted in that 14 day period?

The thread does not appear in the list.

If the thread is restored (after being soft deleted), the thread will once again appear in the list.

 

[Chromaniac]

Thanks. So I assume 14 day is not configurable from backend (would be doing a test installation of 2.2 later tonight). I hope someone would release an add-on to customize this setting!

 

[Chromaniac]

So did some testing on this today. I deleted a thread from similar threads section of another thread. Xenforo replaced that thread with another thread. I wonder if this is desired behavior... Or caching is somehow not working on my board?

 

[Jeremy P]

Up to 100 results are stored in the cache. If one is no longer visible then another will take its place.

 

[dethfire]

How can we change the cache value?

 

[Jeremy P]

You can clear the cache by ticking the box in the Elasticsearch setup page and saving, if that's what you mean.

 

[dethfire]

You can clear the cache by ticking the box in the Elasticsearch setup page and saving, if that's what you mean.

Sorry I mean increase the similar threads cache time

 

[Jeremy P]

It's hard-coded currently, though it might be worth including an option for.

 

[dethfire]

It's hard-coded currently, though it might be worth including an option for.

That would be great! In the meantime, I'm fine changing the file if you have the name/location handy. Thanks!

 

[Jeremy P]

Sure, it's in src/addons/XFES/Entity/ThreadSimilar.php line 24. The 14 is the cache TTL in days.