Pre-Loading Common Search terms.

Slavik

XenForo moderator
Staff member
Taken from here:

http://xenforo.com/community/thread...d-seach-app-yet-kier.24522/page-3#post-314672

Deebs provided me with a shell script to use, however I think it would be good to have this functionality either built in via the addon itself (and option to turn on or off), or a secondary addon linked the the XenForo cron.

What does it do?

When a search is performed, the index file must be loaded from the disk and then executed before it is loaded into the RAM, which may eventually time out and the index needs to be re-loaded. This adds maybe 1-2 seconds onto a search, where as a pre-loaded term (already loaded into the RAM) comes back within 10-20ms

Using this addon, a list of popular searches for your forum, or just a random word list in general could be used to keep those terms consistantly in the RAM, so your popular searches will always come back "instantly" opposed to a wait while the index is loaded and searched.
 
Upvote 9
I haven't dug too much into Elastic Search yet, but it seems like there should be an option to simply preload the *entire* index into RAM. If there isn't, there should be.
From their documentation, the elasticsearch.yml file can have:
Code:
index :
    store:
        type: memory
This means that every index that gets created on the specific node started with the mentioned configuration will store the index in memory. You can also run it as collapsed setting:
Code:
$ elasticsearch -f -Des.index.store.type=memory

The developer does not recommend to use that setting, not sure if this is still valid with latest version. Technically, the total files size should be able to be stored into memory with lose space and can be distributed over several nodes. I believe that is what is referring the developer when he mentions the bugs into his post.

Can anyone let me know what is the indices total size on a 20mil posts board for example?
 
The developer does not recommend to use that setting, not sure if this is still valid with latest version.

As ram is so cheap nowadays, I wonder whether someone tried this option. I have 64 GB RAM and a rather *small* community. If I put the index in ram and restart my server, I guess I would have to re-do the index, right? It is not stored anywhere else than in RAM.
 
As ram is so cheap nowadays, I wonder whether someone tried this option. I have 64 GB RAM and a rather *small* community. If I put the index in ram and restart my server, I guess I would have to re-do the index, right? It is not stored anywhere else than in RAM.

Just use the pre-warmers setting.
 
Top Bottom