XFES\Elasticsearch\RequestException: index already exists

[rfc0001]

After a routine server update to elasticsearch I restarted the server, rebuilt the index (deleting the existing index), then wanted to enable word stemming and stop words (which I had disabled to troubleshoot an issue with searches not returning results), so enabled these and attempted to rebuild the index again (deleting existing) and got error:

XFES\Elasticsearch\RequestException: index [...] already exists in src/addons/XFES/Elasticsearch/Api.php at line 387

1. XFES\Elasticsearch\Api->request() in src/addons/XFES/Elasticsearch/Api.php at line 287
2. XFES\Elasticsearch\Api->createIndex() in src/addons/XFES/Service/Optimizer.php at line 85
3. XFES\Service\Optimizer->optimize() in src/addons/XFES/Service/Configurer.php at line 67
4. XFES\Service\Configurer->initializeIndex() in src/addons/XFES/Admin/Controller/EnhancedSearch.php at line 129
5. XFES\Admin\Controller\EnhancedSearch->actionSetup() in src/XF/Mvc/Dispatcher.php at line 249
6. XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 88
7. XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 41
8. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 1880
9. XF\App->run() in src/XF.php at line 328
10. XF::runApp() in admin.php at line 13

I've since uninstalled/reinstalled Elastisearch, uninstalled/reinstalled Ehanced Search, and keep getting this error when I attempt to set up Enhanced Search again. Ideas?

 

[rfc0001]

Figured this out...had to manually delete the index in elasticsearch

 

[Mike]

I'm really confused how this can happen exactly, because shortly above this code we check if the index exists and if it does, we delete it. So either we're getting a result saying it doesn't exist or the delete isn't working.

What version of Elasticsearch specifically? How did you manually delete the index? (Did you do something like that before?)

Do you have multiple machines in your Elasticsearch cluster?

 

[rfc0001]

I'm really confused how this can happen exactly, because shortly above this code we check if the index exists and if it does, we delete it. So either we're getting a result saying it doesn't exist or the delete isn't working.

What version of Elasticsearch specifically? How did you manually delete the index? (Did you do something like that before?)

Do you have multiple machines in your Elasticsearch cluster?

Elasticsearch 6.1.2

To manually delete an index in Elasticsearch you run following as root:

curl -XDELETE 'localhost:9200/indexname?pretty'

I had not done something like this before.

Note: there were all sorts of java errors in the elasticsearch log in response to the indexname being passed in. My guess is XF got an unexpected response or no response.

The repro on this (not sure if it's consistent since I don't really want to try and repor it ) is rebuild index (with delete) through ACP >tools > rebuild cache, then immediately change ES settings to enable stemming, default stop words, and remove accents and try to rebuild again (with delete). My guess is Elastic search was still busy updating the index when I went to do it a second time.

 

[rfc0001]

For the sake of science I tried to repro this and got a successfull repro:

1. In ACP > Settings > Elasticsearch disable stop words, word stemming, and remove accent marks
2. Rebuild index (with delete)
3. Immediately go back to ACP > Settings > Elasticsearch and enable default stop words, word stemming, and remove acccents marks
4. Rebuild index (with delete)

Boom goes the dynamite:

XFES\Elasticsearch\RequestException: index [indexname/4seTps3OTxO1wLKXvvIggw] already exists src/addons/XFES/Elasticsearch/Api.php:387
Generated by: Russ Jan 18, 2018 at 11:47 AM

Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(287): XFES\Elasticsearch\Api->request('put', 'indexname', '{\n    "mappings...')
#1 src/addons/XFES/Service/Optimizer.php(85): XFES\Elasticsearch\Api->createIndex(Array)
#2 src/addons/XFES/Search/Source/Elasticsearch.php(139): XFES\Service\Optimizer->optimize(Array, true)
#3 src/XF/Search/Search.php(127): XFES\Search\Source\Elasticsearch->truncate('')
#4 src/XF/Job/SearchRebuild.php(25): XF\Search\Search->truncate('')
#5 src/XF/Job/Manager.php(241): XF\Job\SearchRebuild->run(7.9999990463257)
#6 src/XF/Job/Manager.php(187): XF\Job\Manager->runJobInternal(Array, 7.9999990463257)
#7 src/XF/Job/Manager.php(103): XF\Job\Manager->runJobEntry(Array, 7.9999990463257)
#8 src/XF/Admin/Controller/Tools.php(120): XF\Job\Manager->runByIds(Array, 8)
#9 src/XF/Mvc/Dispatcher.php(249): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#10 src/XF/Mvc/Dispatcher.php(88): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', 'html', Object(XF\Mvc\ParameterBag), 'tools', Object(XF\Admin\Controller\Tools), NULL)
#11 src/XF/Mvc/Dispatcher.php(41): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#12 src/XF/App.php(1880): XF\Mvc\Dispatcher->run()
#13 src/XF.php(328): XF\App->run()
#14 admin.php(13): XF::runApp('XF\\Admin\\App')
#15 {main}

 

[rfc0001]

Error in elasticsearch log (truncated since was several thousand characters long:

[2018-01-18T09:47:17,233][WARN ][r.suppressed             ] path: /indexname, params: {index=indexname}
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map
...
                at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
        Caused by: java.io.IOException: Unclosed object or array found
                at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.close(JsonXContentGenerator.java:425) ~[elasticsearch-6.1.2.jar:6.1.2]
                at org.elasticsearch.common.xcontent.XContentBuilder.close(XContentBuilder.java:1034) ~[elasticsearch-6.1.2.jar:6.1.2]
                ... 75 more

 

[Mike]

I'll make a note of this, though I'm not sure we can fix it. It sounds like you may have triggered an internal Elasticsearch issue (possibly with some sort of background rebuild).

 

[rfc0001]

I'll make a note of this, though I'm not sure we can fix it. It sounds like you may have triggered an internal Elasticsearch issue (possibly with some sort of background rebuild).

I am less concerned with the issue itself than the inability to recreate the index even after a uninstall/reinstall of of elasticsearch and enhanced search. Seems like you should be able to check if the index already exists in this condition (or handle the exception that it does gracefully and delete it gracefully and delete it). Thanks.

 

[AndyB]

You might like this add-on. It shows all the Elasticsearch indexes and has a delete option.

https://xenforo.com/community/resources/elasticsearch.5872/

 

[Juanfer]

Hi guys,

I had the same problem and solved it like this:

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/indice/_settings -d '{"index.blocks.read_only_allow_delete": false}'

The problem occurs because the index exceeds more than 90% of the capacity of the partition. And in that case elasticsearch automatically enters into read only mode.

 

[rfc0001]

Sounds like XF is trying to delete it but it's read only, so should be checking if it's read only first? @AndyB is this what your add-on does, which successfully deletes the index in this condition?

 

[Juanfer]

You can simply rebuild the index by removing the previous one, all from the ACP.

 

[rfc0001]

You can simply rebuild the index by removing the previous one, all from the ACP.

Perhaps normally, but not in this case. I completely uninstalled Enhanced Search and Elastic Search and was still getting the error after re-install. I had to manually delete the index per above (or using @AndyB's add-on)