• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

ES 1.1.0

MattW

Well-known member
#1
upload_2014-6-17_11-32-38.png

So, I've created a scripts directory inside the directory where elasticsearch.yaml is located.

I've copied over the xf-date-weighted.mvel to the scripts directory

Code:
[root@host scripts]# ls
xf-date-weighted.mvel
[root@host scripts]# pwd
/etc/elasticsearch/scripts
[root@host scripts]#
I also have this in elasticsearch.yaml
Code:
script.disable_dynamic: false
Good to go?
 

MattW

Well-known member
#2
I can also see this in my ES log
Code:
[2014-06-17 11:16:10,140][INFO ][script                   ] [MWS Host] compiling script file [/etc/elasticsearch/scripts/xf-date-weighted.mvel]
 

Mike

XenForo developer
Staff member
#3
Since you're using dynamic scripting, you don't need to copy the file over. The file method is for non-dynamic scripting.

The non-dynamic version is probably marginally faster, as the script will only be compiled once, rather than once per search.
 

MattW

Well-known member
#4
Hmm, so I removed the script.disable_dynamic: false setting, and restarted ES. The script is loaded, but search still fails.

Code:
2014-06-17 11:46:23,230][INFO ][node                     ] [MWS Host] stopping ...
[2014-06-17 11:46:23,258][INFO ][node                     ] [MWS Host] stopped
[2014-06-17 11:46:23,258][INFO ][node                     ] [MWS Host] closing ...
[2014-06-17 11:46:23,264][INFO ][node                     ] [MWS Host] closed
[2014-06-17 11:46:23,674][INFO ][node                     ] [MWS Host] version[1.2.1], pid[29456], build[6c95b75/2014-06-03T15:02:52Z]
[2014-06-17 11:46:23,675][INFO ][node                     ] [MWS Host] initializing ...
[2014-06-17 11:46:23,678][INFO ][plugins                  ] [MWS Host] loaded [], sites []
[2014-06-17 11:46:25,354][INFO ][script                   ] [MWS Host] compiling script file [/etc/elasticsearch/scripts/xf-date-weighted.mvel]
[2014-06-17 11:46:25,575][INFO ][node                     ] [MWS Host] initialized
[2014-06-17 11:46:25,575][INFO ][node                     ] [MWS Host] starting ...
[2014-06-17 11:46:25,621][INFO ][transport                ] [MWS Host] bound_address {inet[/127.0.0.1:9300]}, publish_address {inet[/127.0.0.1:9300]}
[2014-06-17 11:46:28,639][INFO ][cluster.service          ] [MWS Host] new_master [MWS Host][7xUw6_I0TwqBpccybFoM-g][host.mattwservices.co.uk][inet[/127.0.0.1:9300]], reason: zen-disco-join (elected_as_master)
[2014-06-17 11:46:28,671][INFO ][discovery                ] [MWS Host] mattwservices/7xUw6_I0TwqBpccybFoM-g
[2014-06-17 11:46:28,722][INFO ][http                     ] [MWS Host] bound_address {inet[/127.0.0.1:9200]}, publish_address {inet[/127.0.0.1:9200]}
[2014-06-17 11:46:29,237][INFO ][gateway                  ] [MWS Host] recovered [3] indices into cluster_state
[2014-06-17 11:46:29,242][INFO ][node                     ] [MWS Host] started

Code:
XenForo_Exception: Elasticsearch error: SearchPhaseExecutionException[Failed to execute phase [query_fetch], all shards failed; shardFailures {[7xUw6_I0TwqBpccybFoM-g][mattwservices][0]: SearchParseException[[mattwservices][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"function_score":{"boost_mode":"replace","query":{"query_string":{"query":"nginx","fields":["title^3","message"],"default_operator":"and"}},"script_score":{"params":{"now":1403001991,"halflife":31536000},"script":"_score \/ pow(2.0f, min(10.0f * halflife, abs(now - doc['date'].value + .0f)) \/ (halflife + .0f))"}}},"sort":[{"_score":"desc"},{"date":"desc"}],"size":200,"fields":["discussion_id","user","date"]}]]]; nested: QueryParsingException[[mattwservices] script_score the script could not be loaded]; nested: ScriptException[dynamic scripting disabled]; }] - library/XenES/Search/SourceHandler/ElasticSearch.php:828
Generated By: Matt, A moment ago
Stack Trace
#0 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Search/SourceHandler/ElasticSearch.php(288): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError(Object(stdClass), true)
#1 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/SourceHandler/Abstract.php(115): XenES_Search_SourceHandler_ElasticSearch->executeSearch('nginx', false, Array, Array, false, '200')
#2 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/Searcher.php(79): XenForo_Search_SourceHandler_Abstract->searchGeneral('nginx', Array, 'relevance', '200')
#3 /home/nginx/domains/mattwservices.co.uk/public/library/Andy/SearchLog/ControllerPublic/Search.php(203): XenForo_Search_Searcher->searchGeneral('nginx', Array, 'relevance')
#4 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Proxy/ControllerSearch.php(30): Andy_SearchLog_ControllerPublic_Search->actionSearch()
#5 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(347): XenES_Proxy_ControllerSearch->actionSearch()
#6 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#7 /home/nginx/domains/mattwservices.co.uk/public/index.php(13): XenForo_FrontController->run()
#8 {main}
Request State
array(3) {
  ["url"] => string(41) "https://mattwservices.co.uk/search/search"
  ["_GET"] => array(1) {
    ["/search/search"] => string(0) ""
  }
  ["_POST"] => array(4) {
    ["keywords"] => string(5) "nginx"
    ["users"] => string(0) ""
    ["date"] => string(0) ""
    ["_xfToken"] => string(8) "********"
  }
}
Using Elasticsearch 1.2.1
 

Mike

XenForo developer
Staff member
#5
If you disable dynamic scripting in ES, you need to disable the option mentioned in the first post (and copy the file into the correct scripts directory).

Unfortunately, there's no way for us to detect if dynamic scripting is enabled via the ES API (as far as I can tell).
 

D.O.A.

Well-known member
#6
This appears to be some new form of ES search witchcraft I'm unfamiliar with, and I think I'll stick with the beta until I have running now, until someone layman's it lol. Kthanxbye.
 

MattW

Well-known member
#7
If you disable dynamic scripting in ES, you need to disable the option mentioned in the first post (and copy the file into the correct scripts directory).

Unfortunately, there's no way for us to detect if dynamic scripting is enabled via the ES API (as far as I can tell).
Perfect! Missed that bit
 

RoldanLT

Well-known member
#8
Since you're using dynamic scripting, you don't need to copy the file over. The file method is for non-dynamic scripting.

The non-dynamic version is probably marginally faster, as the script will only be compiled once, rather than once per search.
Thanks for the tip Mike :)
Already upgraded mine with disabled dynamic scripting.
 

Coop1979

Well-known member
#9
This appears to be some new form of ES search witchcraft I'm unfamiliar with, and I think I'll stick with the beta until I have running now, until someone layman's it lol. Kthanxbye.
Yeah, there's something about the way this is being explained that is confusing as all get out.
 

Mike

XenForo developer
Staff member
#11
Yeah, there's something about the way this is being explained that is confusing as all get out.
Unfortunately, we're limited by the underlying behavior of Elasticsearch (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html). The change was made for security reasons, but unfortunately they have done it in a way that is quite limiting/challenging when it comes to distribution of code that needs custom scripting.

As such, it's actually a big backwards compatibility issue. It now requires active configuration to use any feature that involves custom scripting and makes me reticent to use it (and it will certainly not be something we can ever do by default).
 

Xon

Well-known member
#13
I don't get it. What exactly is the advantage of using dynamic scripting in ES?
It allows for custom scripts to be executed per searched item. For XenForo this allows the relevancy weighting by making newer posts ranked higher.

You could also use it to making longer posts ranked higher too.