1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

ES 1.1.0

Discussion in 'Enhanced Search Support' started by MattW, Jun 17, 2014.

  1. MattW

    MattW Well-Known Member

    upload_2014-6-17_11-32-38.png

    So, I've created a scripts directory inside the directory where elasticsearch.yaml is located.

    I've copied over the xf-date-weighted.mvel to the scripts directory

    Code:
    [root@host scripts]# ls
    xf-date-weighted.mvel
    [root@host scripts]# pwd
    /etc/elasticsearch/scripts
    [root@host scripts]# 
    I also have this in elasticsearch.yaml
    Code:
    script.disable_dynamic: false
    Good to go?
     
  2. MattW

    MattW Well-Known Member

    I can also see this in my ES log
    Code:
    [2014-06-17 11:16:10,140][INFO ][script                   ] [MWS Host] compiling script file [/etc/elasticsearch/scripts/xf-date-weighted.mvel]
     
  3. Mike

    Mike XenForo Developer Staff Member

    Since you're using dynamic scripting, you don't need to copy the file over. The file method is for non-dynamic scripting.

    The non-dynamic version is probably marginally faster, as the script will only be compiled once, rather than once per search.
     
    MattW likes this.
  4. MattW

    MattW Well-Known Member

    Hmm, so I removed the script.disable_dynamic: false setting, and restarted ES. The script is loaded, but search still fails.

    Code:
    2014-06-17 11:46:23,230][INFO ][node                     ] [MWS Host] stopping ...
    [2014-06-17 11:46:23,258][INFO ][node                     ] [MWS Host] stopped
    [2014-06-17 11:46:23,258][INFO ][node                     ] [MWS Host] closing ...
    [2014-06-17 11:46:23,264][INFO ][node                     ] [MWS Host] closed
    [2014-06-17 11:46:23,674][INFO ][node                     ] [MWS Host] version[1.2.1], pid[29456], build[6c95b75/2014-06-03T15:02:52Z]
    [2014-06-17 11:46:23,675][INFO ][node                     ] [MWS Host] initializing ...
    [2014-06-17 11:46:23,678][INFO ][plugins                  ] [MWS Host] loaded [], sites []
    [2014-06-17 11:46:25,354][INFO ][script                   ] [MWS Host] compiling script file [/etc/elasticsearch/scripts/xf-date-weighted.mvel]
    [2014-06-17 11:46:25,575][INFO ][node                     ] [MWS Host] initialized
    [2014-06-17 11:46:25,575][INFO ][node                     ] [MWS Host] starting ...
    [2014-06-17 11:46:25,621][INFO ][transport                ] [MWS Host] bound_address {inet[/127.0.0.1:9300]}, publish_address {inet[/127.0.0.1:9300]}
    [2014-06-17 11:46:28,639][INFO ][cluster.service          ] [MWS Host] new_master [MWS Host][7xUw6_I0TwqBpccybFoM-g][host.mattwservices.co.uk][inet[/127.0.0.1:9300]], reason: zen-disco-join (elected_as_master)
    [2014-06-17 11:46:28,671][INFO ][discovery                ] [MWS Host] mattwservices/7xUw6_I0TwqBpccybFoM-g
    [2014-06-17 11:46:28,722][INFO ][http                     ] [MWS Host] bound_address {inet[/127.0.0.1:9200]}, publish_address {inet[/127.0.0.1:9200]}
    [2014-06-17 11:46:29,237][INFO ][gateway                  ] [MWS Host] recovered [3] indices into cluster_state
    [2014-06-17 11:46:29,242][INFO ][node                     ] [MWS Host] started

    Code:
    XenForo_Exception: Elasticsearch error: SearchPhaseExecutionException[Failed to execute phase [query_fetch], all shards failed; shardFailures {[7xUw6_I0TwqBpccybFoM-g][mattwservices][0]: SearchParseException[[mattwservices][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{"query":{"function_score":{"boost_mode":"replace","query":{"query_string":{"query":"nginx","fields":["title^3","message"],"default_operator":"and"}},"script_score":{"params":{"now":1403001991,"halflife":31536000},"script":"_score \/ pow(2.0f, min(10.0f * halflife, abs(now - doc['date'].value + .0f)) \/ (halflife + .0f))"}}},"sort":[{"_score":"desc"},{"date":"desc"}],"size":200,"fields":["discussion_id","user","date"]}]]]; nested: QueryParsingException[[mattwservices] script_score the script could not be loaded]; nested: ScriptException[dynamic scripting disabled]; }] - library/XenES/Search/SourceHandler/ElasticSearch.php:828
    Generated By: Matt, A moment ago
    Stack Trace
    #0 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Search/SourceHandler/ElasticSearch.php(288): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError(Object(stdClass), true)
    #1 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/SourceHandler/Abstract.php(115): XenES_Search_SourceHandler_ElasticSearch->executeSearch('nginx', false, Array, Array, false, '200')
    #2 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/Searcher.php(79): XenForo_Search_SourceHandler_Abstract->searchGeneral('nginx', Array, 'relevance', '200')
    #3 /home/nginx/domains/mattwservices.co.uk/public/library/Andy/SearchLog/ControllerPublic/Search.php(203): XenForo_Search_Searcher->searchGeneral('nginx', Array, 'relevance')
    #4 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Proxy/ControllerSearch.php(30): Andy_SearchLog_ControllerPublic_Search->actionSearch()
    #5 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(347): XenES_Proxy_ControllerSearch->actionSearch()
    #6 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
    #7 /home/nginx/domains/mattwservices.co.uk/public/index.php(13): XenForo_FrontController->run()
    #8 {main}
    Request State
    array(3) {
      ["url"] => string(41) "https://mattwservices.co.uk/search/search"
      ["_GET"] => array(1) {
        ["/search/search"] => string(0) ""
      }
      ["_POST"] => array(4) {
        ["keywords"] => string(5) "nginx"
        ["users"] => string(0) ""
        ["date"] => string(0) ""
        ["_xfToken"] => string(8) "********"
      }
    }
    Using Elasticsearch 1.2.1
     
  5. Mike

    Mike XenForo Developer Staff Member

    If you disable dynamic scripting in ES, you need to disable the option mentioned in the first post (and copy the file into the correct scripts directory).

    Unfortunately, there's no way for us to detect if dynamic scripting is enabled via the ES API (as far as I can tell).
     
    MattW likes this.
  6. D.O.A.

    D.O.A. Well-Known Member

    This appears to be some new form of ES search witchcraft I'm unfamiliar with, and I think I'll stick with the beta until I have running now, until someone layman's it lol. Kthanxbye.
     
    Alfa1 and Coop1979 like this.
  7. MattW

    MattW Well-Known Member

    Perfect! Missed that bit
     
  8. RoldanLT

    RoldanLT Well-Known Member

    Thanks for the tip Mike :)
    Already upgraded mine with disabled dynamic scripting.
     
  9. Coop1979

    Coop1979 Well-Known Member

    Yeah, there's something about the way this is being explained that is confusing as all get out.
     
  10. RoldanLT

    RoldanLT Well-Known Member

    It is very well explained on Admin > Search Option.
    Once you upgraded your ES addon.
     
  11. Mike

    Mike XenForo Developer Staff Member

    Unfortunately, we're limited by the underlying behavior of Elasticsearch (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html). The change was made for security reasons, but unfortunately they have done it in a way that is quite limiting/challenging when it comes to distribution of code that needs custom scripting.

    As such, it's actually a big backwards compatibility issue. It now requires active configuration to use any feature that involves custom scripting and makes me reticent to use it (and it will certainly not be something we can ever do by default).
     
    Rudy and Coop1979 like this.
  12. DRaver

    DRaver Active Member

    I don't get it. What exactly is the advantage of using dynamic scripting in ES?
     
  13. Xon

    Xon Well-Known Member

    It allows for custom scripts to be executed per searched item. For XenForo this allows the relevancy weighting by making newer posts ranked higher.

    You could also use it to making longer posts ranked higher too.
     
    DRaver likes this.

Share This Page