Elasticsearch indexing failed Elasticsearch error: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]

[Alpha1]

I am getting a lot of these errors:

XenForo_Exception: Elasticsearch indexing failed Elasticsearch error: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]; - library/XenES/Search/SourceHandler/ElasticSearch.php:1062
Generated By: Alfa, 36 minutes ago

Stack Trace
#0 /library/XenES/Search/SourceHandler/ElasticSearch.php(1185): XenES_Search_SourceHandler_ElasticSearch->_logSearchResponseError(Object(stdClass), false, 'Elasticsearch i...')
#1 /library/XenES/Search/SourceHandler/ElasticSearch.php(117): XenES_Search_SourceHandler_ElasticSearch->_triggerFailedIndexError(Object(stdClass), Array, true)
#2 /library/XenForo/Search/Indexer.php(105): XenES_Search_SourceHandler_ElasticSearch->deleteFromIndex('post', Array)
#3 /library/XenForo/Search/DataHandler/Post.php(99): XenForo_Search_Indexer->deleteFromIndex('post', Array)
#4 /library/XenForo/Search/DataHandler/Abstract.php(316): XenForo_Search_DataHandler_Post->_deleteFromIndex(Object(XenForo_Search_Indexer), Array)
#5 /library/XenForo/DataWriter/Discussion.php(731): XenForo_Search_DataHandler_Abstract->deleteFromIndex(Object(XenForo_Search_Indexer), Array)
#6 /library/XenForo/DataWriter/Discussion.php(790): XenForo_DataWriter_Discussion->_deleteFromSearchIndex()
#7 /library/XenForo/DataWriter.php(1793): XenForo_DataWriter_Discussion->_postDelete()
#8 /library/XenForo/Model/InlineMod/Thread.php(219): XenForo_DataWriter->delete()
#9 /library/SV/ReportImprovements/XenForo/Model/InlineMod/Thread.php(10): XenForo_Model_InlineMod_Thread->deleteThreads(Array, Array, NULL, NULL)
#10 /library/XenForo/ControllerPublic/InlineMod/Thread.php(45): SV_ReportImprovements_XenForo_Model_InlineMod_Thread->deleteThreads(Array, Array, NULL)
#11 /library/Siropu/AdsManager/ControllerPublic/InlineModThread.php(49): XenForo_ControllerPublic_InlineMod_Thread->actionDelete()
#12 /library/SV/ReportImprovements/XenForo/ControllerPublic/InlineMod/Thread.php(10): Siropu_AdsManager_ControllerPublic_InlineModThread->actionDelete()
#13 /library/XenForo/FrontController.php(369): SV_ReportImprovements_XenForo_ControllerPublic_InlineMod_Thread->actionDelete()
#14 /library/XenForo/FrontController.php(152): XenForo_FrontController->dispatch(Object(XenForo_RouteMatch))
#15 /index.php(13): XenForo_FrontController->run()
#16 {main}
Request State
array(3) {
["url"] => string(48) "https://forum.com/inline-mod/thread/delete"
["_GET"] => array(0) {
}
["_POST"] => array(7) {
["hard_delete"] => string(1) "1"
["resolve_linked_report"] => string(1) "1"
["save"] => string(14) "Delete Threads"
["threads"] => array(1) {
[0] => string(6) "344854"
}
["_xfConfirm"] => string(1) "1"
["redirect"] => string(45) "https://forum.com/find-new/875227/posts"
["_xfToken"] => string(8) "********"
}
}


What puzzles me is the mention of API.
The above likely has to do with the employing of a Web Application Firewall. (StackPath)
I assume that I need to whitelist something. Does anyone have an idea whats going on?

 

[S Thomas]

Did that just happen randomly or did you change something?

 

[Alpha1]

several things changed:

1. we had a major DDoS
2. server IP changed
3. Web Application Firewall (StackPath) added. (but seems to occur when going around the WAF directly to the origin server)

 

[Recep Baltaş]

Same here. Started happening after updating themes I think. Which themes do you use?

 

[Alpha1]

I added diskspace for Elastic but it doesnt help.

Which themes do you use?

Custom style, but no style updates done.

 

[S Thomas]

Probably something you might want to open a ticket for. Or try this: https://discuss.elastic.co/t/forbidden-12-index-read-only-allow-delete-api/110282/5

 

[Recep Baltaş]

This is a serious issue! I've created a ticket. Let's see.

 

[Recep Baltaş]

OK, I ran out of disk space. That is the issue. But after deleting the old index, I still can't start building a new one...

 

[S Thomas]

I just bumped into the same problem. As per link above I just

PUT _settings
{
"index": {
"blocks": {
"read_only_allow_delete": "false"
}
}
}

did that and rebuilt the index.

 

[Mouth]

I just bumped into the same problem. As per link above I just did that and rebuilt the index.

Or, from linux console/command line ...

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": false}'

 

[beerForo]

Got the errors.

Or, from linux console/command line ...

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": false}'

Trying this, thanks.

Edit: Didn't work

Edit: Upgrading ES then running command did it.

 

[Alpha1]

Upgrading Elastic to the latest version solves it.

 

[beerForo]

Upgrading Elastic to the latest version solves it.

On August 7th this worked. Upgraded, ran command line. Today, the same errors are back!

 

[webbouk]

After extending the disc space rebuild the index but give it a new name in the advanced options

 

[webbouk]

Read my post here and subsequent posts...

Elasticsearch indexing error: Elasticsearch bulk action error first error: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]

I am getting this error after upgrading to 2.1 XFES\Elasticsearch\BulkRequestException: Elasticsearch indexing error: Elasticsearch bulk action error (first error: [page-159] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];) src/addons/XFES/Elasticsearch/Api.php:430...

xenforo.com

 

[beerForo]

Even with space, the rebuild triggers the same error.

 

[webbouk]

See my post above yours

 

[beerForo]

Wow, so far so good. Renaming the index does it! THANKS!

 

[beerForo]

Does the old index get deleted, or how do we do that?

 

[webbouk]

Xon posted... Should be under /var/lib/elasticsearch or similar.