Fixed 2.1.0 Preview 1: Error rebuilding ElasticSearch index

Affected version
2.1.0 Preview 1

Rudy

Well-known member
Went to rebuild the ES index, and this happened. We are running version 2.4.3 of ElasticSearch.

GuzzleHttp\Exception\ClientException: Client error: DELETE [URL]http://127.0.0.1:9200/xxxxxxxxxx/post/654080[/URL] resulted in a 404 Not Found response: {"found":false,"_index":"xxxxxxxxxx","_type":"post","_id":"654080","_version":1,"_shards":{"total":2,"successful":1,"f (truncated...) in src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php at line 113
  1. GuzzleHttp\Exception\RequestException::create() insrc/vendor/guzzlehttp/guzzle/src/Middleware.php at line 66
  2. GuzzleHttp\Middleware::GuzzleHttp\{closure}() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 203
  3. GuzzleHttp\Promise\Promise::callHandler() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 156
  4. GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}() insrc/vendor/guzzlehttp/promises/src/TaskQueue.php at line 47
  5. GuzzleHttp\Promise\TaskQueue->run() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 246
  6. GuzzleHttp\Promise\Promise->invokeWaitFn() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 223
  7. GuzzleHttp\Promise\Promise->waitIfPending() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 267
  8. GuzzleHttp\Promise\Promise->invokeWaitList() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 225
  9. GuzzleHttp\Promise\Promise->waitIfPending() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 62
  10. GuzzleHttp\Promise\Promise->wait() insrc/vendor/guzzlehttp/guzzle/src/Client.php at line 106
  11. GuzzleHttp\Client->send() in src/addons/XFES/Elasticsearch/Api.php at line 353
  12. XFES\Elasticsearch\Api->request() insrc/addons/XFES/Elasticsearch/Api.php at line 314
  13. XFES\Elasticsearch\Api->requestFromIndex() insrc/addons/XFES/Elasticsearch/Api.php at line 332
  14. XFES\Elasticsearch\Api->requestById() insrc/addons/XFES/Elasticsearch/Api.php at line 182
  15. XFES\Elasticsearch\Api->delete() insrc/addons/XFES/Search/Source/Elasticsearch.php at line 103
  16. XFES\Search\Source\Elasticsearch->delete() in src/XF/Search/Search.phpat line 122
  17. XF\Search\Search->delete() in src/XF/Search/Search.php at line 47
  18. XF\Search\Search->index() in src/XF/Search/Search.php at line 59
  19. XF\Search\Search->indexEntities() in src/XF/Search/Search.php at line 85
  20. XF\Search\Search->indexRange() in src/XF/Job/SearchRebuild.php at line57
  21. XF\Job\SearchRebuild->run() in src/XF/Job/Manager.php at line 253
  22. XF\Job\Manager->runJobInternal() in src/XF/Job/Manager.php at line195
  23. XF\Job\Manager->runJobEntry() in src/XF/Job/Manager.php at line 111
  24. XF\Job\Manager->runByIds() in src/XF/Admin/Controller/Tools.php at line 120
  25. XF\Admin\Controller\Tools->actionRunJob() insrc/XF/Mvc/Dispatcher.php at line 321
  26. XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line249
  27. XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.phpat line 100
  28. XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line50
  29. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2155
  30. XF\App->run() in src/XF.php at line 389
  31. XF::runApp() in admin.php at line 13
 
Last edited:

Rudy

Well-known member
Updated XF to Beta 2. Still have the problem. Can't index the site, and I get the exact same error with the same post number.

Very interesting problem, and I found what is tripping it up.

I tried to find the post # 654080 in the forum and had no luck. I looked in the xf_post table and found the post which has thread ID, 37272. I tried viewing that thread with a direct URL, and got this error:

You do not have permission to view this page or perform this action.

I now look at xf_thread. The next thread ID, 37273, same thing--won't display. I try 37271, and the thread displays with no problem. Then I take a look--each of these thread titles that won't load begins with the word "Article:". I look further down in the xf_post table and find another thread with the title: "Article: Welcome to the new CMS." Aha!

Anyone remember what that was from? That was from vB 4.x, where they had a CMS front-end feature, and I was testing it on our private forum. We ended up not using it. I run another query and find six others around the same time period; I found a seventh thread that I created using XenForo in 2016, so, that is not affecting anything.

So, there is something in those CMS-generated "threads" that XF does not like and specifically, the Enhanced Search add-on.

I can easily delete those threads and posts, unless someone here at XF wants to take a look and find out what is causing this. I would think there might be other cases where old unused CMS-generated threads might hang something up. I see nothing in xf_post or xf_thread that is unusual for these threads vs. "normal" threads. (I can also dump the database as evidence.)
 

Brent W

Well-known member
Same issue here but with profile post

GuzzleHttp\Exception\ClientException: Client error: DELETE http://127.0.0.1:9200/girls_/xf/profile_post-1331 resulted in a 404 Not Found response: {"index":"girls","_type":"xf","_id":"profile_post-1331","_version":1,"result":"not_found","_shards":{"total":2,"succes (truncated...) in src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php at line 113
  1. GuzzleHttp\Exception\RequestException::create() in src/vendor/guzzlehttp/guzzle/src/Middleware.php at line 66
  2. GuzzleHttp\Middleware::GuzzleHttp\{closure}() in src/vendor/guzzlehttp/promises/src/Promise.php at line 203
  3. GuzzleHttp\Promise\Promise::callHandler() in src/vendor/guzzlehttp/promises/src/Promise.php at line 156
  4. GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}() in src/vendor/guzzlehttp/promises/src/TaskQueue.php at line 47
  5. GuzzleHttp\Promise\TaskQueue->run() in src/vendor/guzzlehttp/promises/src/Promise.php at line 246
  6. GuzzleHttp\Promise\Promise->invokeWaitFn() in src/vendor/guzzlehttp/promises/src/Promise.php at line 223
  7. GuzzleHttp\Promise\Promise->waitIfPending() in src/vendor/guzzlehttp/promises/src/Promise.php at line 267
  8. GuzzleHttp\Promise\Promise->invokeWaitList() in src/vendor/guzzlehttp/promises/src/Promise.php at line 225
  9. GuzzleHttp\Promise\Promise->waitIfPending() in src/vendor/guzzlehttp/promises/src/Promise.php at line 62
  10. GuzzleHttp\Promise\Promise->wait() in src/vendor/guzzlehttp/guzzle/src/Client.php at line 106
  11. GuzzleHttp\Client->send() in src/addons/XFES/Elasticsearch/Api.php at line 353
  12. XFES\Elasticsearch\Api->request() in src/addons/XFES/Elasticsearch/Api.php at line 314
  13. XFES\Elasticsearch\Api->requestFromIndex() in src/addons/XFES/Elasticsearch/Api.php at line 332
  14. XFES\Elasticsearch\Api->requestById() in src/addons/XFES/Elasticsearch/Api.php at line 182
  15. XFES\Elasticsearch\Api->delete() in src/addons/XFES/Search/Source/Elasticsearch.php at line 103
  16. XFES\Search\Source\Elasticsearch->delete() in src/XF/Search/Search.php at line 122
  17. XF\Search\Search->delete() in src/XF/Search/Search.php at line 47
  18. XF\Search\Search->index() in src/XF/Search/Search.php at line 59
  19. XF\Search\Search->indexEntities() in src/XF/Search/Search.php at line 85
  20. XF\Search\Search->indexRange() in src/XF/Job/SearchRebuild.php at line 57
  21. XF\Job\SearchRebuild->run() in src/XF/Job/Manager.php at line 253
  22. XF\Job\Manager->runJobInternal() in src/XF/Job/Manager.php at line 195
  23. XF\Job\Manager->runJobEntry() in src/XF/Job/Manager.php at line 111
  24. XF\Job\Manager->runByIds() in src/XF/Admin/Controller/Tools.php at line 120
  25. XF\Admin\Controller\Tools->actionRunJob() in src/XF/Mvc/Dispatcher.php at line 321
  26. XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 248
  27. XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.php at line 100
  28. XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 50
  29. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2151
  30. XF\App->run() in src/XF.php at line 392
  31. XF::runApp() in admin.php at line 13
 

Rudy

Well-known member
I should note that when I have tried reindexing, it stops on the exact same post every time, and that's whether I do 500 posts in a batch or 5,000. So in other words, it is not randomly stopping on different post numbers when it hits the error.
 

Brent W

Well-known member
I should note that when I have tried reindexing, it stops on the exact same post every time, and that's whether I do 500 posts in a batch or 5,000. So in other words, it is not randomly stopping on different post numbers when it hits the error.
It stops on that exact profile post for me as well.
 

Delazar

Member
I also have the same problem

GuzzleHttp\Exception\ClientException: Client error: DELETE http://127.0.0.1:9200/rpg22/xf/thread-21718 resulted in a 404 Not Found response: {"_index":"rpg22","_type":"xf","_id":"thread-21718","_version":1,"result":"not_found","_shards":{"total":2,"successful": (truncated...) in src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php at line 113
 

Rudy

Well-known member
I want to get my search working again. I want to delete that post, but on the other hand, I really want to keep it around as evidence, in case there is something in that post which is causing the hangup in indexing. (And who's to say that there won't be another post that causes it to stall? Although if that is the case, that would give us two messages it can't index and, perhaps, more evidence...)

I have no issues dumping the database if they need to look at it. I just hope it's not something tied to our individual configurations of ElasticSearch, as that could make it an issue. I would think if my ES is too out of date, version-wise, wouldn't the add-on throw an error message?

Finally...for anyone else with this message, is there anything unusual about the post it is stopping on? Mine seems to be completely unviewable (even as a super admin) other than via the database.
 

Mike

XenForo developer
Staff member
I've fixed this for the next XFES preview/beta release. I've attached a unified diff of the two sets of changes that are needed. (The first is this direct issue, though the second is likely to come up if you only fix the first part.)
 

Attachments

Rezpkt

New member
I had similar issues and I've applied the above diff. However, still got some issues.



Got some exceptions here


Code:
Stack trace
#0 src/XF/Mvc/Entity/Entity.php(106): XF\Mvc\Entity\Entity->get('first_post_like...')
#1 internal_data/code_cache/templates/l1/s40/public/thread_list_macros.php(273): XF\Mvc\Entity\Entity->offsetGet('first_post_like...')
#2 src/XF/Template/Templater.php(693): XF\Template\Templater->{closure}(Object(XF\Template\Templater), Array, Array)
#3 internal_data/code_cache/templates/l1/s40/public/forum_view.php(163): XF\Template\Templater->callMacro('thread_list_mac...', 'item', Array, Array)
#4 src/XF/Template/Templater.php(1293): XF\Template\Templater->{closure}(Object(XF\Template\Templater), Array)
#5 src/XF/Template/Template.php(24): XF\Template\Templater->renderTemplate('forum_view', Array)
#6 src/XF/Mvc/Renderer/Html.php(48): XF\Template\Template->render()
#7 src/XF/Mvc/Dispatcher.php(418): XF\Mvc\Renderer\Html->renderView('XF:Forum\\View', 'public:forum_vi...', Array)
#8 src/XF/Mvc/Dispatcher.php(400): XF\Mvc\Dispatcher->renderView(Object(XF\Mvc\Renderer\Html), Object(XF\Mvc\Reply\View))
#9 src/XF/Mvc/Dispatcher.php(360): XF\Mvc\Dispatcher->renderReply(Object(XF\Mvc\Renderer\Html), Object(XF\Mvc\Reply\View))
#10 src/XF/Mvc/Dispatcher.php(53): XF\Mvc\Dispatcher->render(Object(XF\Mvc\Reply\View), 'html')
#11 src/XF/App.php(2170): XF\Mvc\Dispatcher->run()
#12 src/XF.php(392): XF\App->run()
#13 index.php(20): XF::runApp('XF\\Pub\\App')
#14 {main}
Request state
array(4) {
  ["url"] => string(28) "/index.php?forums/xxxxxxxxxxxx"
  ["referrer"] => string(53) "https://xxxxxxxxx.com/admin.php?enhanced-search/"
  ["_GET"] => array(1) {
    ["xxxxxxxxxxxxx"] => string(0) ""
  }
  ["_POST"] => array(0) {
  }
}

Few of these.





Code:
Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(171): XFES\Elasticsearch\Api->bulkRequest('{"index":{"_ind...')
#1 src/addons/XFES/Search/Source/Elasticsearch.php(82): XFES\Elasticsearch\Api->indexBulk(Array)
#2 src/addons/XFES/Search/Source/Elasticsearch.php(57): XFES\Search\Source\Elasticsearch->flushBulkIndexing()
#3 src/XF/Search/Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#4 src/XF/Search/Search.php(59): XF\Search\Search->index('post', Object(XF\Entity\Post))
#5 src/XF/Search/Search.php(85): XF\Search\Search->indexEntities('post', Object(XF\Mvc\Entity\ArrayCollection))
#6 src/XF/Job/SearchRebuild.php(57): XF\Search\Search->indexRange('post', 1214373, '5000')
#7 src/XF/Job/Manager.php(253): XF\Job\SearchRebuild->run(G)
#8 src/XF/Job/Manager.php(195): XF\Job\Manager->runJobInternal(Array, G)
#9 src/XF/Job/Manager.php(79): XF\Job\Manager->runJobEntry(Array, G)
#10 src/XF/Admin/Controller/Tools.php(139): XF\Job\Manager->runQueue(true, 8)
#11 src/XF/Mvc/Dispatcher.php(321): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#12 src/XF/Mvc/Dispatcher.php(248): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Tools), NULL)
#13 src/XF/Mvc/Dispatcher.php(100): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Tools), NULL)
#14 src/XF/Mvc/Dispatcher.php(50): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#15 src/XF/App.php(2170): XF\Mvc\Dispatcher->run()
#16 src/XF.php(392): XF\App->run()
#17 admin.php(13): XF::runApp('XF\\Admin\\App')
#18 {main}

array(4) {
  ["url"] => string(24) "/admin.php?tools/run-job"
  ["referrer"] => string(50) "https://xxxxxxxxx.com/admin.php?tools/run-job"
  ["_GET"] => array(1) {
    ["tools/run-job"] => string(0) ""
  }
  ["_POST"] => array(3) {
    ["_xfRedirect"] => string(10) "/admin.php"
    ["_xfToken"] => string(8) "********"
    ["only_ids"] => string(0) ""
  }
}

seems that its stuck, can't finish indexing.

2.1.0Beta4. ES 2.1.0 Preview 1
 
Last edited:

Mike

XenForo developer
Staff member
The first error is unrelated. It likely points to outdated templates.

The second error is very likely to be add-on related, particularly if you have an add-on that allows multiple prefixes to be used. If so (or you have installed it in the past), you would need to contact the author for guidance.
 

Rezpkt

New member
The first error is unrelated. It likely points to outdated templates.

The second error is very likely to be add-on related, particularly if you have an add-on that allows multiple prefixes to be used. If so (or you have installed it in the past), you would need to contact the author for guidance.
edit: I've fixed the first exception, thanks for the pointer.

Thanks for the swift reply, Mike.

regarding the second:
This is a fresh install, file wise. There's no add-ons installed apart from ES. Database is from V1 which has been upgraded to V2. All add-ons were uninstalled before doing the upgrade. Only the upgraded database has come across to this install.

I'm a developer so I'm happy to try and debug this if you have any pointers. My question is , how can this be happening with a fresh install and no add-ons installed? Does the database hold code which is executed or have any influence on it?
 
Last edited:

Xon

Well-known member
I've seen very old copies of MultiPrefix (including what looks like a poor pirated knock-off) from before I took it over not correctly uninstall leaving the prefix_id column in a bad state.

If you can run show create table xf_thread and see if the prefix_id column is not an integer it is likely an old broken add-on
 

Rezpkt

New member
I've seen very old copies of MultiPrefix (including what looks like a poor pirated knock-off) from before I took it over not correctly uninstall leaving the prefix_id column in a bad state.

If you can run show create table xf_thread and see if the prefix_id column is not an integer it is likely an old broken add-on
Xon, thanks so much for the pointer - it was set to some kind of binary type - I have now set it to integer.

All seems to be working now - I've got it setup on AWS Elasticsearch. Only issue was actually setting the URL - kept getting a guzzle 404 because it was appending the index name to the url which didn't exist yet. I had to do a PUT manually with postman to create the index, and it then worked. Also think it would help if there was a note http/https:// shouldn't be in the URL
 
Top