Fixed 2.1.0 Preview 1: Error rebuilding ElasticSearch index

[Wildcat Media]

Affected version

2.1.0 Preview 1

Went to rebuild the ES index, and this happened. We are running version 2.4.3 of ElasticSearch.

GuzzleHttp\Exception\ClientException: Client error: DELETE [URL]http://127.0.0.1:9200/xxxxxxxxxx/post/654080[/URL] resulted in a 404 Not Found response: {"found":false,"_index":"xxxxxxxxxx","_type":"post","_id":"654080","_version":1,"_shards":{"total":2,"successful":1,"f (truncated...) in src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php at line 113

1. GuzzleHttp\Exception\RequestException::create() insrc/vendor/guzzlehttp/guzzle/src/Middleware.php at line 66
2. GuzzleHttp\Middleware::GuzzleHttp\{closure}() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 203
3. GuzzleHttp\Promise\Promise::callHandler() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 156
4. GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}() insrc/vendor/guzzlehttp/promises/src/TaskQueue.php at line 47
5. GuzzleHttp\Promise\TaskQueue->run() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 246
6. GuzzleHttp\Promise\Promise->invokeWaitFn() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 223
7. GuzzleHttp\Promise\Promise->waitIfPending() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 267
8. GuzzleHttp\Promise\Promise->invokeWaitList() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 225
9. GuzzleHttp\Promise\Promise->waitIfPending() insrc/vendor/guzzlehttp/promises/src/Promise.php at line 62
10. GuzzleHttp\Promise\Promise->wait() insrc/vendor/guzzlehttp/guzzle/src/Client.php at line 106
11. GuzzleHttp\Client->send() in src/addons/XFES/Elasticsearch/Api.php at line 353
12. XFES\Elasticsearch\Api->request() insrc/addons/XFES/Elasticsearch/Api.php at line 314
13. XFES\Elasticsearch\Api->requestFromIndex() insrc/addons/XFES/Elasticsearch/Api.php at line 332
14. XFES\Elasticsearch\Api->requestById() insrc/addons/XFES/Elasticsearch/Api.php at line 182
15. XFES\Elasticsearch\Api->delete() insrc/addons/XFES/Search/Source/Elasticsearch.php at line 103
16. XFES\Search\Source\Elasticsearch->delete() in src/XF/Search/Search.phpat line 122
17. XF\Search\Search->delete() in src/XF/Search/Search.php at line 47
18. XF\Search\Search->index() in src/XF/Search/Search.php at line 59
19. XF\Search\Search->indexEntities() in src/XF/Search/Search.php at line 85
20. XF\Search\Search->indexRange() in src/XF/Job/SearchRebuild.php at line57
21. XF\Job\SearchRebuild->run() in src/XF/Job/Manager.php at line 253
22. XF\Job\Manager->runJobInternal() in src/XF/Job/Manager.php at line195
23. XF\Job\Manager->runJobEntry() in src/XF/Job/Manager.php at line 111
24. XF\Job\Manager->runByIds() in src/XF/Admin/Controller/Tools.php at line 120
25. XF\Admin\Controller\Tools->actionRunJob() insrc/XF/Mvc/Dispatcher.php at line 321
26. XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line249
27. XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.phpat line 100
28. XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line50
29. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2155
30. XF\App->run() in src/XF.php at line 389
31. XF::runApp() in admin.php at line 13

 

[Wildcat Media]

Updated XF to Beta 2. Still have the problem. Can't index the site, and I get the exact same error with the same post number.

Very interesting problem, and I found what is tripping it up.

I tried to find the post # 654080 in the forum and had no luck. I looked in the xf_post table and found the post which has thread ID, 37272. I tried viewing that thread with a direct URL, and got this error:

You do not have permission to view this page or perform this action.

I now look at xf_thread. The next thread ID, 37273, same thing--won't display. I try 37271, and the thread displays with no problem. Then I take a look--each of these thread titles that won't load begins with the word "Article:". I look further down in the xf_post table and find another thread with the title: "Article: Welcome to the new CMS." Aha!

Anyone remember what that was from? That was from vB 4.x, where they had a CMS front-end feature, and I was testing it on our private forum. We ended up not using it. I run another query and find six others around the same time period; I found a seventh thread that I created using XenForo in 2016, so, that is not affecting anything.

So, there is something in those CMS-generated "threads" that XF does not like and specifically, the Enhanced Search add-on.

I can easily delete those threads and posts, unless someone here at XF wants to take a look and find out what is causing this. I would think there might be other cases where old unused CMS-generated threads might hang something up. I see nothing in xf_post or xf_thread that is unusual for these threads vs. "normal" threads. (I can also dump the database as evidence.)

 

[Brent W]

Same issue here but with profile post

GuzzleHttp\Exception\ClientException: Client error: DELETE http://127.0.0.1:9200/girls_/xf/profile_post-1331 resulted in a 404 Not Found response: {"index":"girls","_type":"xf","_id":"profile_post-1331","_version":1,"result":"not_found","_shards":{"total":2,"succes (truncated...) in src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php at line 113

1. GuzzleHttp\Exception\RequestException::create() in src/vendor/guzzlehttp/guzzle/src/Middleware.php at line 66
2. GuzzleHttp\Middleware::GuzzleHttp\{closure}() in src/vendor/guzzlehttp/promises/src/Promise.php at line 203
3. GuzzleHttp\Promise\Promise::callHandler() in src/vendor/guzzlehttp/promises/src/Promise.php at line 156
4. GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}() in src/vendor/guzzlehttp/promises/src/TaskQueue.php at line 47
5. GuzzleHttp\Promise\TaskQueue->run() in src/vendor/guzzlehttp/promises/src/Promise.php at line 246
6. GuzzleHttp\Promise\Promise->invokeWaitFn() in src/vendor/guzzlehttp/promises/src/Promise.php at line 223
7. GuzzleHttp\Promise\Promise->waitIfPending() in src/vendor/guzzlehttp/promises/src/Promise.php at line 267
8. GuzzleHttp\Promise\Promise->invokeWaitList() in src/vendor/guzzlehttp/promises/src/Promise.php at line 225
9. GuzzleHttp\Promise\Promise->waitIfPending() in src/vendor/guzzlehttp/promises/src/Promise.php at line 62
10. GuzzleHttp\Promise\Promise->wait() in src/vendor/guzzlehttp/guzzle/src/Client.php at line 106
11. GuzzleHttp\Client->send() in src/addons/XFES/Elasticsearch/Api.php at line 353
12. XFES\Elasticsearch\Api->request() in src/addons/XFES/Elasticsearch/Api.php at line 314
13. XFES\Elasticsearch\Api->requestFromIndex() in src/addons/XFES/Elasticsearch/Api.php at line 332
14. XFES\Elasticsearch\Api->requestById() in src/addons/XFES/Elasticsearch/Api.php at line 182
15. XFES\Elasticsearch\Api->delete() in src/addons/XFES/Search/Source/Elasticsearch.php at line 103
16. XFES\Search\Source\Elasticsearch->delete() in src/XF/Search/Search.php at line 122
17. XF\Search\Search->delete() in src/XF/Search/Search.php at line 47
18. XF\Search\Search->index() in src/XF/Search/Search.php at line 59
19. XF\Search\Search->indexEntities() in src/XF/Search/Search.php at line 85
20. XF\Search\Search->indexRange() in src/XF/Job/SearchRebuild.php at line 57
21. XF\Job\SearchRebuild->run() in src/XF/Job/Manager.php at line 253
22. XF\Job\Manager->runJobInternal() in src/XF/Job/Manager.php at line 195
23. XF\Job\Manager->runJobEntry() in src/XF/Job/Manager.php at line 111
24. XF\Job\Manager->runByIds() in src/XF/Admin/Controller/Tools.php at line 120
25. XF\Admin\Controller\Tools->actionRunJob() in src/XF/Mvc/Dispatcher.php at line 321
26. XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 248
27. XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.php at line 100
28. XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 50
29. XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2151
30. XF\App->run() in src/XF.php at line 392
31. XF::runApp() in admin.php at line 13

 

[Wildcat Media]

I should note that when I have tried reindexing, it stops on the exact same post every time, and that's whether I do 500 posts in a batch or 5,000. So in other words, it is not randomly stopping on different post numbers when it hits the error.

 

[Brent W]

I should note that when I have tried reindexing, it stops on the exact same post every time, and that's whether I do 500 posts in a batch or 5,000. So in other words, it is not randomly stopping on different post numbers when it hits the error.

It stops on that exact profile post for me as well.

 

[Delazar]

I also have the same problem

GuzzleHttp\Exception\ClientException: Client error: DELETE http://127.0.0.1:9200/rpg22/xf/thread-21718 resulted in a 404 Not Found response: {"_index":"rpg22","_type":"xf","_id":"thread-21718","_version":1,"result":"not_found","_shards":{"total":2,"successful": (truncated...) in src/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php at line 113

 

[Wildcat Media]

I want to get my search working again. I want to delete that post, but on the other hand, I really want to keep it around as evidence, in case there is something in that post which is causing the hangup in indexing. (And who's to say that there won't be another post that causes it to stall? Although if that is the case, that would give us two messages it can't index and, perhaps, more evidence...)

I have no issues dumping the database if they need to look at it. I just hope it's not something tied to our individual configurations of ElasticSearch, as that could make it an issue. I would think if my ES is too out of date, version-wise, wouldn't the add-on throw an error message?

Finally...for anyone else with this message, is there anything unusual about the post it is stopping on? Mine seems to be completely unviewable (even as a super admin) other than via the database.

 

[Mike]

I've fixed this for the next XFES preview/beta release. I've attached a unified diff of the two sets of changes that are needed. (The first is this direct issue, though the second is likely to come up if you only fix the first part.)

 

[Wildcat Media]

Worked perfectly--thanks Mike!

 

[Rezpkt]

I had similar issues and I've applied the above diff. However, still got some issues.



Got some exceptions here

Accessed unknown getter 'like_count' on XF:User[134948] 9 minutes ago src/XF/Mvc/Entity/Entity.php:188

Accessed unknown getter 'first_post_likes' on XF:Thread[768329] 9 minutes ago src/XF/Mvc/Entity/Entity.php:188

Stack trace
#0 src/XF/Mvc/Entity/Entity.php(106): XF\Mvc\Entity\Entity->get('first_post_like...')
#1 internal_data/code_cache/templates/l1/s40/public/thread_list_macros.php(273): XF\Mvc\Entity\Entity->offsetGet('first_post_like...')
#2 src/XF/Template/Templater.php(693): XF\Template\Templater->{closure}(Object(XF\Template\Templater), Array, Array)
#3 internal_data/code_cache/templates/l1/s40/public/forum_view.php(163): XF\Template\Templater->callMacro('thread_list_mac...', 'item', Array, Array)
#4 src/XF/Template/Templater.php(1293): XF\Template\Templater->{closure}(Object(XF\Template\Templater), Array)
#5 src/XF/Template/Template.php(24): XF\Template\Templater->renderTemplate('forum_view', Array)
#6 src/XF/Mvc/Renderer/Html.php(48): XF\Template\Template->render()
#7 src/XF/Mvc/Dispatcher.php(418): XF\Mvc\Renderer\Html->renderView('XF:Forum\\View', 'public:forum_vi...', Array)
#8 src/XF/Mvc/Dispatcher.php(400): XF\Mvc\Dispatcher->renderView(Object(XF\Mvc\Renderer\Html), Object(XF\Mvc\Reply\View))
#9 src/XF/Mvc/Dispatcher.php(360): XF\Mvc\Dispatcher->renderReply(Object(XF\Mvc\Renderer\Html), Object(XF\Mvc\Reply\View))
#10 src/XF/Mvc/Dispatcher.php(53): XF\Mvc\Dispatcher->render(Object(XF\Mvc\Reply\View), 'html')
#11 src/XF/App.php(2170): XF\Mvc\Dispatcher->run()
#12 src/XF.php(392): XF\App->run()
#13 index.php(20): XF::runApp('XF\\Pub\\App')
#14 {main}
Request state
array(4) {
  ["url"] => string(28) "/index.php?forums/xxxxxxxxxxxx"
  ["referrer"] => string(53) "https://xxxxxxxxx.com/admin.php?enhanced-search/"
  ["_GET"] => array(1) {
    ["xxxxxxxxxxxxx"] => string(0) ""
  }
  ["_POST"] => array(0) {
  }
}

Few of these.

Elasticsearch indexing error: Elasticsearch bulk action error (first error: [thread-102705] failed to parse [prefix]) 8 minutes ago src/addons/XFES/Elasticsearch/Api.php:408

Stack trace
#0 src/addons/XFES/Elasticsearch/Api.php(171): XFES\Elasticsearch\Api->bulkRequest('{"index":{"_ind...')
#1 src/addons/XFES/Search/Source/Elasticsearch.php(82): XFES\Elasticsearch\Api->indexBulk(Array)
#2 src/addons/XFES/Search/Source/Elasticsearch.php(57): XFES\Search\Source\Elasticsearch->flushBulkIndexing()
#3 src/XF/Search/Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#4 src/XF/Search/Search.php(59): XF\Search\Search->index('post', Object(XF\Entity\Post))
#5 src/XF/Search/Search.php(85): XF\Search\Search->indexEntities('post', Object(XF\Mvc\Entity\ArrayCollection))
#6 src/XF/Job/SearchRebuild.php(57): XF\Search\Search->indexRange('post', 1214373, '5000')
#7 src/XF/Job/Manager.php(253): XF\Job\SearchRebuild->run(G)
#8 src/XF/Job/Manager.php(195): XF\Job\Manager->runJobInternal(Array, G)
#9 src/XF/Job/Manager.php(79): XF\Job\Manager->runJobEntry(Array, G)
#10 src/XF/Admin/Controller/Tools.php(139): XF\Job\Manager->runQueue(true, 8)
#11 src/XF/Mvc/Dispatcher.php(321): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#12 src/XF/Mvc/Dispatcher.php(248): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Tools), NULL)
#13 src/XF/Mvc/Dispatcher.php(100): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Tools), NULL)
#14 src/XF/Mvc/Dispatcher.php(50): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#15 src/XF/App.php(2170): XF\Mvc\Dispatcher->run()
#16 src/XF.php(392): XF\App->run()
#17 admin.php(13): XF::runApp('XF\\Admin\\App')
#18 {main}

array(4) {
  ["url"] => string(24) "/admin.php?tools/run-job"
  ["referrer"] => string(50) "https://xxxxxxxxx.com/admin.php?tools/run-job"
  ["_GET"] => array(1) {
    ["tools/run-job"] => string(0) ""
  }
  ["_POST"] => array(3) {
    ["_xfRedirect"] => string(10) "/admin.php"
    ["_xfToken"] => string(8) "********"
    ["only_ids"] => string(0) ""
  }
}

seems that its stuck, can't finish indexing.

2.1.0Beta4. ES 2.1.0 Preview 1

 

[Mike]

The first error is unrelated. It likely points to outdated templates.

The second error is very likely to be add-on related, particularly if you have an add-on that allows multiple prefixes to be used. If so (or you have installed it in the past), you would need to contact the author for guidance.

 

[Rezpkt]

The first error is unrelated. It likely points to outdated templates.

The second error is very likely to be add-on related, particularly if you have an add-on that allows multiple prefixes to be used. If so (or you have installed it in the past), you would need to contact the author for guidance.

edit: I've fixed the first exception, thanks for the pointer.

Thanks for the swift reply, Mike.

regarding the second:
This is a fresh install, file wise. There's no add-ons installed apart from ES. Database is from V1 which has been upgraded to V2. All add-ons were uninstalled before doing the upgrade. Only the upgraded database has come across to this install.

I'm a developer so I'm happy to try and debug this if you have any pointers. My question is , how can this be happening with a fresh install and no add-ons installed? Does the database hold code which is executed or have any influence on it?

 

[Xon]

I've seen very old copies of MultiPrefix (including what looks like a poor pirated knock-off) from before I took it over not correctly uninstall leaving the prefix_id column in a bad state.

If you can run show create table xf_thread and see if the prefix_id column is not an integer it is likely an old broken add-on

 

[Rezpkt]

I've seen very old copies of MultiPrefix (including what looks like a poor pirated knock-off) from before I took it over not correctly uninstall leaving the prefix_id column in a bad state.

If you can run show create table xf_thread and see if the prefix_id column is not an integer it is likely an old broken add-on

Xon, thanks so much for the pointer - it was set to some kind of binary type - I have now set it to integer.

All seems to be working now - I've got it setup on AWS Elasticsearch. Only issue was actually setting the URL - kept getting a guzzle 404 because it was appending the index name to the url which didn't exist yet. I had to do a PUT manually with postman to create the index, and it then worked. Also think it would help if there was a note http/https:// shouldn't be in the URL