ElasticSearch Essentials [Paid] 3.17.14

ivp · Jan 18, 2023

ivp said:
Is it possible to use ES for user mentions and Find member field at https://xenforo.com/community/members/

Xon said:
It has been something I've been wanting to implement but haven't had the time. Likely sometime after the XF2.2 update

@Xon any update on this?

Xon · Jan 18, 2023

ivp said:
@Xon any update on this?

I've ended up releasing this as a separate add-on that doesn't depend on ElasticSearch Essentials.

Username auto-complete using XFES

Requires XenForo Enhanced Search (XFES) and Search Improvements v2.5.3+ to actually do anything! Replace the stock MySQL-powered username autocomplete with Elasticsearch powered version. Supports partial matching, non-prefix search. Autocomplete...

xenforo.com

It works a lot nicer for username autocomplete due to being typo-resistant and supporting non-prefix lookups.

A similar add-on for tag autocomplete:

Tag auto-complete using XFES

Requires XenForo Enhanced Search (XFES) and Search Improvements v2.5.3+ to actually do anything! Replace the stock MySQL-powered tag autocomplete with Elasticsearch powered version. Supports partial matching, non-prefix search. Autocomplete is...

xenforo.com

Gossipy · Jan 18, 2023

Good day,

We are having trouble with this addon.

Whenever we activate any of the following options:

Weight by contents reactions
Weight by contents views
Weight by reply count (threads)

A full-text search by a user will generate a server error: "Elasticsearch error: field value function must not produce negative scores".

Strangely, it doesn't seem to happen if the user selects "search titles only". Full error at bottom of post.

User will receive a message "this search could not be completed".

It happens with any one of the above options active, and with all of them active. Once we put all three of the above options back to 0 and rebuild the index, the error stops and search works for users again.

Changing "Content weighting score mode" from "sum" to "average" does not fix.

Changing the weighting numbers on the above options does not fix.

XF version is 2.2.12
XFES version is 2.2.2
Add-on version is 3.12.3
ElasticSearch version is 7.17.8
PHP version is 8.1.14

Full error:

Code:

XFES\Elasticsearch\RequestException: Elasticsearch error: field value function must not produce negative scores, but got: [-5.5000001192092896] for field value: [-5.0] src/addons/XFES/Elasticsearch/Api.php:424
Generated by: User007 Jan 18, 2023 at 4:14 PM
Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(338): XFES\Elasticsearch\Api->request('get', 'db_name/_s...', '{
    "sort": [...')
#1 src/addons/XFES/Elasticsearch/Api.php(236): XFES\Elasticsearch\Api->requestFromIndex('get', '_search', Array)
#2 src/addons/XFES/Search/Source/Elasticsearch.php(750): XFES\Elasticsearch\Api->search(Array)
#3 src/addons/XFES/Search/Source/Elasticsearch.php(146): XFES\Search\Source\Elasticsearch->executeSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, '200')
#4 src/XF/Search/Search.php(185): XFES\Search\Source\Elasticsearch->search(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), '200')
#5 src/XF/Search/Search.php(214): XF\Search\Search->XF\Search\{closure}(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), '200')
#6 src/XF/Search/Search.php(181): XF\Search\Search->executeSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), '200', Object(Closure), true)
#7 src/XF/Repository/Search.php(33): XF\Search\Search->search(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery))
#8 src/addons/SV/ElasticSearchEssentials/XF/Repository/Search.php(39): XF\Repository\Search->runSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#9 src/addons/SV/ElasticSearchEssentials/Repository/SearchTrait.php(245): SV\ElasticSearchEssentials\XF\Repository\Search->SV\ElasticSearchEssentials\XF\Repository\{closure}(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#10 src/addons/SV/ElasticSearchEssentials/XF/Repository/Search.php(35): SV\ElasticSearchEssentials\XF\Repository\Search->wrapRunSearchForSvEss(Object(Closure), Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#11 src/addons/SV/ElasticSearchEssentials/XF/Pub/Controller/Search.php(648): SV\ElasticSearchEssentials\XF\Repository\Search->runSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#12 src/XF/Pub/Controller/Search.php(119): SV\ElasticSearchEssentials\XF\Pub\Controller\Search->runSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array)
#13 src/addons/SV/ElasticSearchEssentials/XF/Pub/Controller/Search.php(481): XF\Pub\Controller\Search->actionSearch()
#14 src/XF/Mvc/Dispatcher.php(352): SV\ElasticSearchEssentials\XF\Pub\Controller\Search->actionSearch(Object(XF\Mvc\ParameterBag))
#15 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Search', 'Search', Object(XF\Mvc\RouteMatch), Object(SV\ElasticSearchEssentials\XF\Pub\Controller\Search), NULL)
#16 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\ElasticSearchEssentials\XF\Pub\Controller\Search), NULL)
#17 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#18 src/XF/App.php(2483): XF\Mvc\Dispatcher->run()
#19 src/XF.php(524): XF\App->run()
#20 index.php(20): XF::runApp('XF\\Pub\\App')
#21 {main}
Request state
array(4) {
  ["url"] => string(24) "/community/search/search"
  ["referrer"] => string(58) "https://www.oursite.com/community/online/?type=member"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(3) {
    ["keywords"] => string(5) "fruit"
    ["c"] => array(1) {
      ["users"] => string(0) ""
    }
    ["_xfToken"] => string(8) "********"
  }
}

Thanks in advance for any help.

Xon · Jan 19, 2023

@Gossipy Can you ensure you've got "Search Improvements" installed and have the option "On error, log search DSL" set. Otherwise debugging this is next to impossible.

You can open a ticket on my site with the details as this will dump a the entire query that is sent to ElasticSearch

Gossipy · Jan 19, 2023

Xon said:
@Gossipy Can you ensure you've got "Search Improvements" installed and have the option "On error, log search DSL" set. Otherwise debugging this is next to impossible.

I don't think we have "Search Improvements" installed - I'll check into that now and follow your suggestion.

Xon said:
You can open a ticket on my site with the details as this will dump a the entire query that is sent to ElasticSearch

Thank you, will do.

Xon · Feb 14, 2023

Xon updated ElasticSearch Essentials with a new update entry:

3.13.0 - Feature update

Improve detection of 'empty results' for various search result options (Search Improvements v2.8.0+ compatibility)

Add "Collapsible similar content" style property option (for this add-on's widget), allowing the similar thread box to be collapsed by the user and remembered via a cookie

Adjust various search option defaults:

Autocomplete sort type: Relevance

Enable Autocomplete for mobiles: true

Apply content weights to auto-complete: true

Weight by reply...

Read the rest of this update entry...

ivp · Mar 5, 2023

Noticed that post count by forum in "Post Areas" does not match count provided using SQL query:

Code:

SELECT
    xf_node.title,
    COUNT(*) AS total
FROM
    xf_post,
    xf_thread,
    xf_node,
    xf_forum
WHERE
    xf_post.user_id = SOMEUSERID AND xf_post.message_state = "visible" AND xf_post.thread_id = xf_thread.thread_id AND xf_node.node_id = xf_thread.node_id AND xf_node.node_id = xf_forum.node_id AND xf_forum.count_messages = 1
GROUP BY
    xf_thread.node_id
ORDER BY
    total
DESC

SQL query displays more posts for almost every forum.

Also total number in "Post Areas" is different than "Message" number on the profile page.

Please advise.

Xon · Mar 5, 2023

This is the result of the add-on reading the xf_thread_user_post table. This is vastly faster and avoids touching the post table.

Older versions of XF 1.x didn't correctly populate this table. These cache values can be rebuild via the 'rebuild threads' with the 'post positions' option selected.

ivp · Mar 5, 2023

Will try php cmd.php xf-rebuild:threads --position_rebuild. Thanks.

ivp · Mar 6, 2023

It finished after 2.5 hours, but the change is negligible, still having the same issue.

Also tried https://xenforo.com/community/resources/liam-w-rip-post-count-rebuild.5817/ and it's the same.

I compared results from the previous SQL query with the one fetching data from xf_thread_user_post and the results are the same:

Code:

SELECT
    xf_node.title,
    SUM(post_count) AS total
FROM
    xf_thread_user_post,
    xf_thread,
    xf_node,
    xf_forum
WHERE
    xf_thread_user_post.user_id = SOMEUSERID AND xf_thread_user_post.thread_id = xf_thread.thread_id AND xf_node.node_id = xf_thread.node_id AND xf_node.node_id = xf_forum.node_id AND xf_forum.count_messages = 1
GROUP BY
    xf_thread.node_id
ORDER BY
    total
DESC

"Post Areas" tab displays different figures. It even does not show one of the forums in the list.

Also confusing part is that "Message" number on the profile page is different than total number in "Post Areas" and different from SQL queries.

Xon · Mar 6, 2023

Your query is missing if the thread is visible, and isn't checking the forum allows viewing threads by other users. It is a little more complex than a simple count all posts by forum for a user.

ivp · Mar 6, 2023

Since I am administrator, I should have a privilege to see it all?

I've added xf_thread.discussion_state="visible" to query and now number from SQL (177.840) matches the number Messages on profile (177.840).

But still seeing discrepancy in total number of messages (178.429) at the bottom of "Post Areas". I think it ignores xf_forum.count_messages = 1, but still not getting that number when removing that clause.

Regarding missing forum in the list, it doesn't display forum having xf_node.display_in_list = 0, so this is actually fine.

Xon · Mar 8, 2023

Xon updated ElasticSearch Essentials with a new update entry:

3.13.1 - Bugfix update

Improve autocomplete results by excluding forums where the user lacks "view content" permission (aka they can see titles but not content).

Fix "Enable suggestions only for the first page of search results" was not applying for low search result count

Harden displaying "no results found" for search results (do not risk using cached search results).
This add-on is not vulnerable to leaking cached search results like Search Improvements v2.8.0/v2.8.1 did.
However if...

Read the rest of this update entry...

HybridBoy · Mar 9, 2023

@Xon

I have surely misconfigured this addon and I need your help to fix it.

I have upgraded to the latest version, and rebuilt the index.

Basically, when creating a new thread title let's say a review for a company named: "Namecheap.com Review" it would only appear in "Similar threads" if creating the thread is in the same forum category.

If a thread was created in "another forum category" it would not show in "similar threads" that it was already posted.

And second issue:
I perform a search from homepage , so i type in in search box, let's say I search the company "Namecheap.com Review" it does not show it was already posted,

but if i search the same "Namecheap.com Review" in the forum category where this company thread is already posted quick results show.

Xon · Mar 10, 2023

It sounds like the permissions for the similar thread and autocomplete feature haven't been granted (or where revoked). For anything more in-depth please open a ticket on my site.

Xon · Mar 10, 2023

I think the last update had a regression in how it selected forums for auto-complete/similar threads. Looking into it now.

Xon · Mar 10, 2023

Xon updated ElasticSearch Essentials with a new update entry:

3.13.2 - Bugfix update

Fix regression in v3.13.1 causing most forums to be skipped from autocomplete/similar search

Read the rest of this update entry...

Xon · Mar 10, 2023

HybridBoy said:
Basically, when creating a new thread title let's say a review for a company named: "Namecheap.com Review" it would only appear in "Similar threads" if creating the thread is in the same forum category.

If a thread was created in "another forum category" it would not show in "similar threads" that it was already posted.

The default action is to only suggest similar threads (or autocomplete results) in the same forum and subforums. Ie it deliberately doesn't show threads for everywhere.

You can change this by editing each node or using this SQL to edit all nodes;

SQL:

update xf_forum
set ess_limit_autocomplete = 'all', ess_limit_suggest = 'all';

Once this query is run, you'll need to edit any node/forum and change the display order. You can set it back after, but this is required to invalidate a cache.

HybridBoy · Mar 10, 2023

Thank you for helping me configure the addon to its full potential. Super happy! TOP-NOTCH SUPPORT!

Xon · Mar 12, 2023

Xon updated ElasticSearch Essentials with a new update entry:

3.13.3 - Bugfix update

Fix "Redirect to top validated suggestion" option was inverted.

Fix "Redirect to top validated suggestion" could redirect to non-validated suggestions if "Validate Suggestions" was disabled

Read the rest of this update entry...

ElasticSearch Essentials [Paid] 3.17.14

Active member

Well-known member

Active member

Well-known member

Active member

Well-known member

Active member

Well-known member

Active member

Active member

Well-known member

Active member

Well-known member

Active member

Well-known member

Well-known member

Well-known member

Well-known member

Active member

Well-known member

Similar threads

We value your privacy