Elasticsearch indexing error: [parent] Data too large, data for [<http_request>] would be [123647988/117.9mb]

[sbj]

Hello,

for first time I am trying out ES right now. I got it installed on my localhost to try it out on a safe environment first.

I got this error while rebuilding the search index. It seems some kind of limit stopped the rebuilding. I just don't know what limit is exceeded.
Does anyone know what I have to increase so it works for indexing?
There is also a mention of an addon here, but I believe it is not the cause of that.

Thanks.

edit:

XF: 2.1.3
XFES. 2.1.2
ES: 7.4.2
MariaDB: 10.1.32

Server error log

    XFES\Elasticsearch\RequestException: Elasticsearch indexing error: [parent] Data too large, data for [<http_request>] would be [123647988/117.9mb], which is larger than the limit of [123273216/117.5mb], real usage: [123067056/117.3mb], new bytes reserved: [580932/567.3kb], usages [request=0/0b, fielddata=0/0b, in_flight_requests=580932/567.3kb, accounting=14260/13.9kb] src\addons\XFES\Elasticsearch\Api.php:409

    Generated by: xxx 9 November 2019 at 22:34

Stack trace

#0 src\addons\XFES\Elasticsearch\Api.php(425): XFES\Elasticsearch\Api->request('post', '_bulk', '{"index":{"_ind...', Array, Object(GuzzleHttp\Psr7\Request))
#1 src\addons\XFES\Elasticsearch\Api.php(180): XFES\Elasticsearch\Api->bulkRequest('{"index":{"_ind...')
#2 src\addons\XFES\Search\Source\Elasticsearch.php(82): XFES\Elasticsearch\Api->indexBulk(Array)
#3 src\addons\XFES\Search\Source\Elasticsearch.php(57): XFES\Search\Source\Elasticsearch->flushBulkIndexing()
#4 src\XF\Search\Search.php(40): XFES\Search\Source\Elasticsearch->index(Object(XF\Search\IndexRecord))
#5 src\XF\Search\Search.php(59): XF\Search\Search->index('thread_field', Object(AL\ThreadFilter\Entity\ThreadFieldIndex))
#6 src\XF\Search\Search.php(85): XF\Search\Search->indexEntities('thread_field', Object(XF\Mvc\Entity\ArrayCollection))
#7 src\XF\Job\SearchRebuild.php(57): XF\Search\Search->indexRange('thread_field', 2219697, '500')
#8 src\XF\Job\Manager.php(253): XF\Job\SearchRebuild->run(G)
#9 src\XF\Job\Manager.php(195): XF\Job\Manager->runJobInternal(Array, G)
#10 src\XF\Job\Manager.php(111): XF\Job\Manager->runJobEntry(Array, G)
#11 src\XF\Admin\Controller\Tools.php(120): XF\Job\Manager->runByIds(Array, 8)
#12 src\XF\Mvc\Dispatcher.php(321): XF\Admin\Controller\Tools->actionRunJob(Object(XF\Mvc\ParameterBag))
#13 src\XF\Mvc\Dispatcher.php(248): XF\Mvc\Dispatcher->dispatchClass('XF:Tools', 'RunJob', Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Tools), NULL)
#14 src\XF\Mvc\Dispatcher.php(100): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\Tools), NULL)
#15 src\XF\Mvc\Dispatcher.php(50): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#16 src\XF\App.php(2178): XF\Mvc\Dispatcher->run()
#17 src\XF.php(390): XF\App->run()
#18 admin.php(13): XF::runApp('XF\\Admin\\App')
#19 {main}

Request state

array(4) {
  ["url"] => string(24) "/admin.php?tools/run-job"
  ["referrer"] => string(40) "http://localhost/admin.php?tools/run-job"
  ["_GET"] => array(1) {
    ["tools/run-job"] => string(0) ""
  }
  ["_POST"] => array(3) {
    ["_xfRedirect"] => string(50) "http://localhost/admin.php?tools/rebuild&success=1"
    ["_xfToken"] => string(8) "********"
    ["only_ids"] => string(4) "4848"
  }
}

 

[Xon]

Try lowering the batch size, it is trying to push +115mb into ElasticSearch at once (must have some really big posts)

 

[sbj]

That didn't help. But I found out the solution. I just had to increase the limit for ES to

-Xms256m
-Xmx256m

I thought I did that before I posted here but it turns out I forgot to save it in the file, so it had no effect. Stupid me, sorry.