ElasticSearch Essentials

ElasticSearch Essentials [Paid] 3.15.12

No permission to buy ($30.00)
Overview FAQ Updates (85) Reviews (15) History Discussion
ivp said:
@Xon any update on this?
Click to expand...
I've ended up releasing this as a separate add-on that doesn't depend on ElasticSearch Essentials.
xenforo.com

Username auto-complete using XFES

Requires XenForo Enhanced Search (XFES) and Search Improvements v2.5.3+ to actually do anything! Replace the stock MySQL-powered username autocomplete with Elasticsearch powered version. Supports partial matching, non-prefix search. Autocomplete...
xenforo.com xenforo.com

It works a lot nicer for username autocomplete due to being typo-resistant and supporting non-prefix lookups.

A similar add-on for tag autocomplete:
xenforo.com

Tag auto-complete using XFES

Requires XenForo Enhanced Search (XFES) and Search Improvements v2.5.3+ to actually do anything! Replace the stock MySQL-powered tag autocomplete with Elasticsearch powered version. Supports partial matching, non-prefix search. Autocomplete is...
xenforo.com xenforo.com
 
Good day,

We are having trouble with this addon.

Whenever we activate any of the following options:
  • Weight by contents reactions
  • Weight by contents views
  • Weight by reply count (threads)
A full-text search by a user will generate a server error: "Elasticsearch error: field value function must not produce negative scores".

Strangely, it doesn't seem to happen if the user selects "search titles only". Full error at bottom of post.

User will receive a message "this search could not be completed".

It happens with any one of the above options active, and with all of them active. Once we put all three of the above options back to 0 and rebuild the index, the error stops and search works for users again.

Changing "Content weighting score mode" from "sum" to "average" does not fix.

Changing the weighting numbers on the above options does not fix.

XF version is 2.2.12
XFES version is 2.2.2
Add-on version is 3.12.3
ElasticSearch version is 7.17.8
PHP version is 8.1.14

Full error:
Code: 
XFES\Elasticsearch\RequestException: Elasticsearch error: field value function must not produce negative scores, but got: [-5.5000001192092896] for field value: [-5.0] src/addons/XFES/Elasticsearch/Api.php:424
Generated by: User007 Jan 18, 2023 at 4:14 PM
Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(338): XFES\Elasticsearch\Api->request('get', 'db_name/_s...', '{
    "sort": [...')
#1 src/addons/XFES/Elasticsearch/Api.php(236): XFES\Elasticsearch\Api->requestFromIndex('get', '_search', Array)
#2 src/addons/XFES/Search/Source/Elasticsearch.php(750): XFES\Elasticsearch\Api->search(Array)
#3 src/addons/XFES/Search/Source/Elasticsearch.php(146): XFES\Search\Source\Elasticsearch->executeSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, '200')
#4 src/XF/Search/Search.php(185): XFES\Search\Source\Elasticsearch->search(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), '200')
#5 src/XF/Search/Search.php(214): XF\Search\Search->XF\Search\{closure}(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), '200')
#6 src/XF/Search/Search.php(181): XF\Search\Search->executeSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), '200', Object(Closure), true)
#7 src/XF/Repository/Search.php(33): XF\Search\Search->search(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery))
#8 src/addons/SV/ElasticSearchEssentials/XF/Repository/Search.php(39): XF\Repository\Search->runSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#9 src/addons/SV/ElasticSearchEssentials/Repository/SearchTrait.php(245): SV\ElasticSearchEssentials\XF\Repository\Search->SV\ElasticSearchEssentials\XF\Repository\{closure}(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#10 src/addons/SV/ElasticSearchEssentials/XF/Repository/Search.php(35): SV\ElasticSearchEssentials\XF\Repository\Search->wrapRunSearchForSvEss(Object(Closure), Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#11 src/addons/SV/ElasticSearchEssentials/XF/Pub/Controller/Search.php(648): SV\ElasticSearchEssentials\XF\Repository\Search->runSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array, true)
#12 src/XF/Pub/Controller/Search.php(119): SV\ElasticSearchEssentials\XF\Pub\Controller\Search->runSearch(Object(SV\ElasticSearchEssentials\XF\Search\Query\KeywordQuery), Array)
#13 src/addons/SV/ElasticSearchEssentials/XF/Pub/Controller/Search.php(481): XF\Pub\Controller\Search->actionSearch()
#14 src/XF/Mvc/Dispatcher.php(352): SV\ElasticSearchEssentials\XF\Pub\Controller\Search->actionSearch(Object(XF\Mvc\ParameterBag))
#15 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('XF:Search', 'Search', Object(XF\Mvc\RouteMatch), Object(SV\ElasticSearchEssentials\XF\Pub\Controller\Search), NULL)
#16 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\ElasticSearchEssentials\XF\Pub\Controller\Search), NULL)
#17 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#18 src/XF/App.php(2483): XF\Mvc\Dispatcher->run()
#19 src/XF.php(524): XF\App->run()
#20 index.php(20): XF::runApp('XF\\Pub\\App')
#21 {main}
Request state
array(4) {
  ["url"] => string(24) "/community/search/search"
  ["referrer"] => string(58) "https://www.oursite.com/community/online/?type=member"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(3) {
    ["keywords"] => string(5) "fruit"
    ["c"] => array(1) {
      ["users"] => string(0) ""
    }
    ["_xfToken"] => string(8) "********"
  }
}

Thanks in advance for any help.
 
Last edited:
@Gossipy Can you ensure you've got "Search Improvements" installed and have the option "On error, log search DSL" set. Otherwise debugging this is next to impossible.

You can open a ticket on my site with the details as this will dump a the entire query that is sent to ElasticSearch
 
Xon said:
@Gossipy Can you ensure you've got "Search Improvements" installed and have the option "On error, log search DSL" set. Otherwise debugging this is next to impossible.
Click to expand...

I don't think we have "Search Improvements" installed - I'll check into that now and follow your suggestion.

Xon said:
You can open a ticket on my site with the details as this will dump a the entire query that is sent to ElasticSearch
Click to expand...

Thank you, will do.
 
  • Like
Reactions: Xon
Xon updated ElasticSearch Essentials with a new update entry:

3.13.0 - Feature update

  • Improve detection of 'empty results' for various search result options (Search Improvements v2.8.0+ compatibility)
  • Add "Collapsible similar content" style property option (for this add-on's widget), allowing the similar thread box to be collapsed by the user and remembered via a cookie
  • Adjust various search option defaults:
    • Autocomplete sort type: Relevance
    • Enable Autocomplete for mobiles: true
    • Apply content weights to auto-complete: true
    • Weight by reply...
Click to expand...

Read the rest of this update entry...
 
Noticed that post count by forum in "Post Areas" does not match count provided using SQL query:
Code: 
SELECT
    xf_node.title,
    COUNT(*) AS total
FROM
    xf_post,
    xf_thread,
    xf_node,
    xf_forum
WHERE
    xf_post.user_id = SOMEUSERID AND xf_post.message_state = "visible" AND xf_post.thread_id = xf_thread.thread_id AND xf_node.node_id = xf_thread.node_id AND xf_node.node_id = xf_forum.node_id AND xf_forum.count_messages = 1
GROUP BY
    xf_thread.node_id
ORDER BY
    total
DESC
SQL query displays more posts for almost every forum.

Also total number in "Post Areas" is different than "Message" number on the profile page.

Please advise.
 
This is the result of the add-on reading the xf_thread_user_post table. This is vastly faster and avoids touching the post table.

Older versions of XF 1.x didn't correctly populate this table. These cache values can be rebuild via the 'rebuild threads' with the 'post positions' option selected.
 
It finished after 2.5 hours, but the change is negligible, still having the same issue.

Also tried https://xenforo.com/community/resources/liam-w-rip-post-count-rebuild.5817/ and it's the same.

I compared results from the previous SQL query with the one fetching data from xf_thread_user_post and the results are the same:
Code: 
SELECT
    xf_node.title,
    SUM(post_count) AS total
FROM
    xf_thread_user_post,
    xf_thread,
    xf_node,
    xf_forum
WHERE
    xf_thread_user_post.user_id = SOMEUSERID AND xf_thread_user_post.thread_id = xf_thread.thread_id AND xf_node.node_id = xf_thread.node_id AND xf_node.node_id = xf_forum.node_id AND xf_forum.count_messages = 1
GROUP BY
    xf_thread.node_id
ORDER BY
    total
DESC
"Post Areas" tab displays different figures. It even does not show one of the forums in the list.

Also confusing part is that "Message" number on the profile page is different than total number in "Post Areas" and different from SQL queries.
 
Last edited:
Your query is missing if the thread is visible, and isn't checking the forum allows viewing threads by other users. It is a little more complex than a simple count all posts by forum for a user.
 
Since I am administrator, I should have a privilege to see it all?

I've added xf_thread.discussion_state="visible" to query and now number from SQL (177.840) matches the number Messages on profile (177.840).

But still seeing discrepancy in total number of messages (178.429) at the bottom of "Post Areas". I think it ignores xf_forum.count_messages = 1, but still not getting that number when removing that clause.

Regarding missing forum in the list, it doesn't display forum having xf_node.display_in_list = 0, so this is actually fine.
 
Xon updated ElasticSearch Essentials with a new update entry:

3.13.1 - Bugfix update

  • Improve autocomplete results by excluding forums where the user lacks "view content" permission (aka they can see titles but not content).
  • Fix "Enable suggestions only for the first page of search results" was not applying for low search result count
  • Harden displaying "no results found" for search results (do not risk using cached search results).
    This add-on is not vulnerable to leaking cached search results like Search Improvements v2.8.0/v2.8.1 did.
    However if...
Click to expand...

Read the rest of this update entry...
 
@Xon

I have surely misconfigured this addon and I need your help to fix it.

I have upgraded to the latest version, and rebuilt the index.

Basically, when creating a new thread title let's say a review for a company named: "Namecheap.com Review" it would only appear in "Similar threads" if creating the thread is in the same forum category.

If a thread was created in "another forum category" it would not show in "similar threads" that it was already posted.

And second issue:
I perform a search from homepage , so i type in in search box, let's say I search the company "Namecheap.com Review" it does not show it was already posted,

but if i search the same "Namecheap.com Review" in the forum category where this company thread is already posted quick results show.
 
I think the last update had a regression in how it selected forums for auto-complete/similar threads. Looking into it now.
 
HybridBoy said:
Basically, when creating a new thread title let's say a review for a company named: "Namecheap.com Review" it would only appear in "Similar threads" if creating the thread is in the same forum category.

If a thread was created in "another forum category" it would not show in "similar threads" that it was already posted.
Click to expand...
The default action is to only suggest similar threads (or autocomplete results) in the same forum and subforums. Ie it deliberately doesn't show threads for everywhere.

You can change this by editing each node or using this SQL to edit all nodes;
SQL: 
update xf_forum
set ess_limit_autocomplete = 'all', ess_limit_suggest = 'all';
Once this query is run, you'll need to edit any node/forum and change the display order. You can set it back after, but this is required to invalidate a cache.
 
Last edited:
You must log in or register to reply here.

Similar threads

GameNet
XFES\Elasticsearch\RequestException: Elasticsearch error: all shards failed src/addons/XFES/Elasticsearch/Api.php:464
Replies
6
Views
139
GameNet
GameNet
CihanOnline
Duplicate Error After Enabling Search Suggestions - XFES Elasticsearch Configuration Issue
Replies
1
Views
73
Jeremy P
Jeremy P
H
  • Question Question
ES 2.2 How To Upgrade Elasticsearch 7.x for Xenforo on CentOS
Replies
0
Views
220
HappyWorld
H
Mr Lucky
  • Question Question
Version 2.3.0 beta 2 XFES\Elasticsearch\RequestException: Elasticsearch indexing error
Replies
8
Views
270
Mr Lucky
Mr Lucky
Joshua
  • Question Question
XF 2.2 elasticsearch is acting up. can i just reinstall?
Replies
1
Views
206
MattW
MattW
Top Bottom