ES 1.1.0

[MattW]

So, I've created a scripts directory inside the directory where elasticsearch.yaml is located.

I've copied over the xf-date-weighted.mvel to the scripts directory

[root@host scripts]# ls
xf-date-weighted.mvel
[root@host scripts]# pwd
/etc/elasticsearch/scripts
[root@host scripts]#

I also have this in elasticsearch.yaml

script.disable_dynamic: false

Good to go?

 

[MattW]

I can also see this in my ES log

[2014-06-17 11:16:10,140][INFO ][script                   ] [MWS Host] compiling script file [/etc/elasticsearch/scripts/xf-date-weighted.mvel]

 

[Mike]

Since you're using dynamic scripting, you don't need to copy the file over. The file method is for non-dynamic scripting.

The non-dynamic version is probably marginally faster, as the script will only be compiled once, rather than once per search.

 

[MattW]

Hmm, so I removed the script.disable_dynamic: false setting, and restarted ES. The script is loaded, but search still fails.

2014-06-17 11:46:23,230][INFO ][node                     ] [MWS Host] stopping ...
[2014-06-17 11:46:23,258][INFO ][node                     ] [MWS Host] stopped
[2014-06-17 11:46:23,258][INFO ][node                     ] [MWS Host] closing ...
[2014-06-17 11:46:23,264][INFO ][node                     ] [MWS Host] closed
[2014-06-17 11:46:23,674][INFO ][node                     ] [MWS Host] version[1.2.1], pid[29456], build[6c95b75/2014-06-03T15:02:52Z]
[2014-06-17 11:46:23,675][INFO ][node                     ] [MWS Host] initializing ...
[2014-06-17 11:46:23,678][INFO ][plugins                  ] [MWS Host] loaded [], sites []
[2014-06-17 11:46:25,354][INFO ][script                   ] [MWS Host] compiling script file [/etc/elasticsearch/scripts/xf-date-weighted.mvel]
[2014-06-17 11:46:25,575][INFO ][node                     ] [MWS Host] initialized
[2014-06-17 11:46:25,575][INFO ][node                     ] [MWS Host] starting ...
[2014-06-17 11:46:25,621][INFO ][transport                ] [MWS Host] bound_address {inet[/127.0.0.1:9300]}, publish_address {inet[/127.0.0.1:9300]}
[2014-06-17 11:46:28,639][INFO ][cluster.service          ] [MWS Host] new_master [MWS Host][7xUw6_I0TwqBpccybFoM-g][host.mattwservices.co.uk][inet[/127.0.0.1:9300]], reason: zen-disco-join (elected_as_master)
[2014-06-17 11:46:28,671][INFO ][discovery                ] [MWS Host] mattwservices/7xUw6_I0TwqBpccybFoM-g
[2014-06-17 11:46:28,722][INFO ][http                     ] [MWS Host] bound_address {inet[/127.0.0.1:9200]}, publish_address {inet[/127.0.0.1:9200]}
[2014-06-17 11:46:29,237][INFO ][gateway                  ] [MWS Host] recovered [3] indices into cluster_state
[2014-06-17 11:46:29,242][INFO ][node                     ] [MWS Host] started

XenForo_Exception: Elasticsearch error: SearchPhaseExecutionException[Failed to execute phase [query_fetch], all shards failed; shardFailures {[7xUw6_I0TwqBpccybFoM-g][mattwservices][0]: SearchParseException[[mattwservices][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"function_score":{"boost_mode":"replace","query":{"query_string":{"query":"nginx","fields":["title^3","message"],"default_operator":"and"}},"script_score":{"params":{"now":1403001991,"halflife":31536000},"script":"_score \/ pow(2.0f, min(10.0f * halflife, abs(now - doc['date'].value + .0f)) \/ (halflife + .0f))"}}},"sort":[{"_score":"desc"},{"date":"desc"}],"size":200,"fields":["discussion_id","user","date"]}]]]; nested: QueryParsingException[[mattwservices] script_score the script could not be loaded]; nested: ScriptException[dynamic scripting disabled]; }] - library/XenES/Search/SourceHandler/ElasticSearch.php:828
Generated By: Matt, A moment ago
Stack Trace
#0 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Search/SourceHandler/ElasticSearch.php(288): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError(Object(stdClass), true)
#1 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/SourceHandler/Abstract.php(115): XenES_Search_SourceHandler_ElasticSearch->executeSearch('nginx', false, Array, Array, false, '200')
#2 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/Searcher.php(79): XenForo_Search_SourceHandler_Abstract->searchGeneral('nginx', Array, 'relevance', '200')
#3 /home/nginx/domains/mattwservices.co.uk/public/library/Andy/SearchLog/ControllerPublic/Search.php(203): XenForo_Search_Searcher->searchGeneral('nginx', Array, 'relevance')
#4 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Proxy/ControllerSearch.php(30): Andy_SearchLog_ControllerPublic_Search->actionSearch()
#5 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(347): XenES_Proxy_ControllerSearch->actionSearch()
#6 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#7 /home/nginx/domains/mattwservices.co.uk/public/index.php(13): XenForo_FrontController->run()
#8 {main}
Request State
array(3) {
  ["url"] => string(41) "https://mattwservices.co.uk/search/search"
  ["_GET"] => array(1) {
    ["/search/search"] => string(0) ""
  }
  ["_POST"] => array(4) {
    ["keywords"] => string(5) "nginx"
    ["users"] => string(0) ""
    ["date"] => string(0) ""
    ["_xfToken"] => string(8) "********"
  }
}

Using Elasticsearch 1.2.1

 

[Mike]

If you disable dynamic scripting in ES, you need to disable the option mentioned in the first post (and copy the file into the correct scripts directory).

Unfortunately, there's no way for us to detect if dynamic scripting is enabled via the ES API (as far as I can tell).

 

[D.O.A.]

This appears to be some new form of ES search witchcraft I'm unfamiliar with, and I think I'll stick with the beta until I have running now, until someone layman's it lol. Kthanxbye.

 

[MattW]

If you disable dynamic scripting in ES, you need to disable the option mentioned in the first post (and copy the file into the correct scripts directory).

Unfortunately, there's no way for us to detect if dynamic scripting is enabled via the ES API (as far as I can tell).

Perfect! Missed that bit

 

[rdn]

Since you're using dynamic scripting, you don't need to copy the file over. The file method is for non-dynamic scripting.

The non-dynamic version is probably marginally faster, as the script will only be compiled once, rather than once per search.

Thanks for the tip Mike
Already upgraded mine with disabled dynamic scripting.

 

[Coop1979]

This appears to be some new form of ES search witchcraft I'm unfamiliar with, and I think I'll stick with the beta until I have running now, until someone layman's it lol. Kthanxbye.

Yeah, there's something about the way this is being explained that is confusing as all get out.

 

[rdn]

Yeah, there's something about the way this is being explained that is confusing as all get out.

It is very well explained on Admin > Search Option.
Once you upgraded your ES addon.

 

[Mike]

Yeah, there's something about the way this is being explained that is confusing as all get out.

Unfortunately, we're limited by the underlying behavior of Elasticsearch (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html). The change was made for security reasons, but unfortunately they have done it in a way that is quite limiting/challenging when it comes to distribution of code that needs custom scripting.

As such, it's actually a big backwards compatibility issue. It now requires active configuration to use any feature that involves custom scripting and makes me reticent to use it (and it will certainly not be something we can ever do by default).

 

[DRaver]

I don't get it. What exactly is the advantage of using dynamic scripting in ES?

 

[Xon]

I don't get it. What exactly is the advantage of using dynamic scripting in ES?

It allows for custom scripts to be executed per searched item. For XenForo this allows the relevancy weighting by making newer posts ranked higher.

You could also use it to making longer posts ranked higher too.