Fixed Elasticsearch 2.0 Error

[MattW]

Just upgraded to ES2.0, and the search no longer works. Index can be rebuilt OK, but searching throws the below error

Error Info
ErrorException: Object of class stdClass could not be converted to string - library/XenES/Search/SourceHandler/ElasticSearch.php:845
Generated By: Matt, 1 minute ago
Stack Trace
#0 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Search/SourceHandler/ElasticSearch.php(845): XenForo_Application::handlePhpError(4096, 'Object of class...', '/home/nginx/dom...', 845, Array)
#1 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Search/SourceHandler/ElasticSearch.php(305): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError(Object(stdClass), true)
#2 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/SourceHandler/Abstract.php(115): XenES_Search_SourceHandler_ElasticSearch->executeSearch('nginx', false, Array, Array, false, '200')
#3 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/Searcher.php(79): XenForo_Search_SourceHandler_Abstract->searchGeneral('nginx', Array, 'relevance', '200')
#4 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/ControllerPublic/Search.php(246): XenForo_Search_Searcher->searchGeneral('nginx', Array, 'relevance')
#5 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Proxy/ControllerSearch.php(30): XenForo_ControllerPublic_Search->actionSearch()
#6 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(351): XenES_Proxy_ControllerSearch->actionSearch()
#7 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#8 /home/nginx/domains/mattwservices.co.uk/public/index.php(13): XenForo_FrontController->run()
#9 {main}
Request State
array(3) {
  ["url"] => string(41) "https://mattwservices.co.uk/search/search"
  ["_GET"] => array(1) {
    ["/search/search"] => string(0) ""
  }
  ["_POST"] => array(4) {
    ["keywords"] => string(5) "nginx"
    ["users"] => string(0) ""
    ["date"] => string(0) ""
    ["_xfToken"] => string(8) "********"
  }
}

 

[janowitz]

Did you really upgrade just out of curiosity on your production site or do you urgently need some of the new features? I would prefer to wait for some official announcement on XenForo.com and even then I would first test it on some test environment...

 

[Mike]

I setup a new ES 2.0 install and a new index and didn't have any problems out of the box.

This error is actually in the error handling itself, so something else is the underlying problem. I have managed to force an error and reproduced this particular issue so we can see the original error.

In library/XenES/Search/SourceHandler/ElasticSearch.php, find:

$e = new XenForo_Exception(trim($extraMessage . ' Elasticsearch error: ' . $response->error));

Replace with:

            if (is_string($response->error))
            {
                $e = new XenForo_Exception(trim($extraMessage . ' Elasticsearch error: ' . $response->error));
            }
            else
            {
                $e = new XenForo_Exception(trim($extraMessage . ' Elasticsearch error: ' . json_encode($response->error)));
            }

What is the exception that gets logged then?

 

[MattW]

Thanks Mike

XenForo_Exception: Elasticsearch error: {"root_cause":[{"type":"script_exception","reason":"scripts of type [inline], operation [search] and lang [groovy] are disabled"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"mattwser_xenforo","node":"KoSCe1l4QBGqV5lVDEz5Vw","reason":{"type":"query_parsing_exception","reason":"script_score the script could not be loaded","index":"mattwser_xenforo","line":1,"col":258,"caused_by":{"type":"script_exception","reason":"scripts of type [inline], operation [search] and lang [groovy] are disabled"}}}]} - library/XenES/Search/SourceHandler/ElasticSearch.php:852
Generated By: Matt, A moment ago
Stack Trace
#0 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Search/SourceHandler/ElasticSearch.php(305): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError(Object(stdClass), true)
#1 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/SourceHandler/Abstract.php(115): XenES_Search_SourceHandler_ElasticSearch->executeSearch('nginx', false, Array, Array, false, '200')
#2 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/Searcher.php(79): XenForo_Search_SourceHandler_Abstract->searchGeneral('nginx', Array, 'relevance', '200')
#3 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/ControllerPublic/Search.php(246): XenForo_Search_Searcher->searchGeneral('nginx', Array, 'relevance')
#4 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Proxy/ControllerSearch.php(30): XenForo_ControllerPublic_Search->actionSearch()
#5 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(351): XenES_Proxy_ControllerSearch->actionSearch()
#6 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#7 /home/nginx/domains/mattwservices.co.uk/public/index.php(13): XenForo_FrontController->run()
#8 {main}
Request State
array(3) {
  ["url"] => string(41) "https://mattwservices.co.uk/search/search"
  ["_GET"] => array(1) {
    ["/search/search"] => string(0) ""
  }
  ["_POST"] => array(4) {
    ["keywords"] => string(5) "nginx"
    ["users"] => string(0) ""
    ["date"] => string(0) ""
    ["_xfToken"] => string(8) "********"
  }
}

 

[MattW]

I've been able to get pass the above error by adding the below to elasticsearch.yml

script.inline: on
script.indexed: on

It's now throwing

XenForo_Exception: Elasticsearch error: {"root_cause":[{"type":"groovy_script_execution_exception","reason":"failed to run inline script [xf-date-weighted] using lang [groovy]"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"mattwser_xenforo","node":"eKWqrsPYSqCOimhLCuRYRA","reason":{"type":"groovy_script_execution_exception","reason":"failed to run inline script [xf-date-weighted] using lang [groovy]","caused_by":{"type":"missing_property_exception","reason":"No such property: xf for class: bbd027383bd3132a8f12db815e1f96f3795541ac"}}}]} - library/XenES/Search/SourceHandler/ElasticSearch.php:852
Generated By: Matt, 3 minutes ago
Stack Trace
#0 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Search/SourceHandler/ElasticSearch.php(305): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError(Object(stdClass), true)
#1 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/SourceHandler/Abstract.php(115): XenES_Search_SourceHandler_ElasticSearch->executeSearch('nginx', false, Array, Array, false, '200')
#2 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/Search/Searcher.php(79): XenForo_Search_SourceHandler_Abstract->searchGeneral('nginx', Array, 'relevance', '200')
#3 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/ControllerPublic/Search.php(246): XenForo_Search_Searcher->searchGeneral('nginx', Array, 'relevance')
#4 /home/nginx/domains/mattwservices.co.uk/public/library/XenES/Proxy/ControllerSearch.php(30): XenForo_ControllerPublic_Search->actionSearch()
#5 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(351): XenES_Proxy_ControllerSearch->actionSearch()
#6 /home/nginx/domains/mattwservices.co.uk/public/library/XenForo/FrontController.php(134): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#7 /home/nginx/domains/mattwservices.co.uk/public/index.php(13): XenForo_FrontController->run()
#8 {main}
Request State
array(3) {
  ["url"] => string(41) "https://mattwservices.co.uk/search/search"
  ["_GET"] => array(1) {
    ["/search/search"] => string(0) ""
  }
  ["_POST"] => array(4) {
    ["keywords"] => string(5) "nginx"
    ["users"] => string(0) ""
    ["date"] => string(0) ""
    ["_xfToken"] => string(8) "********"
  }
}

EDIT:
Un-ticking the date weighted box allows it to run

 

[Mike]

That's roughly where I've ended at too as I started to test that.

It seemed that when I used dynamic scripting in XF, it seemed to work. It's just failing to load the script in the Elasticsearch config/scripts directory. (I assume you've copied the XF script there.)

 

[MattW]

That's roughly where I've ended at too as I started to test that.

It seemed that when I used dynamic scripting in XF, it seemed to work. It's just failing to load the script in the Elasticsearch config/scripts directory. (I assume you've copied the XF script there.)

Yes, it's there on both nodes.

 

[Mike]

This relates to changes to Elasticsearch's script APIs in 2.0. I'm attaching a unified diff of the fix.

 

[Daniel Hood]

I might be wrong because I'm looking at a small portion of the file but it seems you could get an undefined variable error with that. There's cases where it seems $scriptType may not be defined but still called if you're on ES2.0

 

[Mike]

The diff isn't easy to read. There are 2 lines that define $scriptType (if and else).

 

[Daniel Hood]

Ah, I didn't really think about

}
-
-                if ($version >= '1.0')

getting removed and resulting in $scriptType = 'file'; ending up in the else. My mistake.

 

[MattW]

This relates to changes to Elasticsearch's script APIs in 2.0. I'm attaching a unified diff of the fix.

Fantastic, thanks Mike. Fix applied and working

 

[estranged]

How should we apply the fix?

Thanks.

 

[Paul B]

Until a new release is available, you will have to manually edit the file.

 

[Mike]

If that isn't viable, the issue doesn't happen when XF uses dynamic scripting. (Or stick with Elasticsearch 1.x for now.)

 

[estranged]

Can you post clearer instructions on how to edit the file please? What to delete or replace etc?

 

[Paul B]

The instructions are in the file as it's a diff file.

The - indicates lines which must be removed, the + indicates lines which must be added.

 

[estranged]

Can someone who has already done this attach the file please?

Thank you.

 

[Jeremy]

If you haven't upgraded to ElasticSearch 2, you should be fine without the patch until the next release.

 

[estranged]

I upgraded to elasticsearch 2. That's why I was asking. Thank you anyway.