Implemented Rebuild Search by CLI

[Deriel]

I just rebuilt the search of my forum (3M posts) and it was fast, but surly it would be faster by CLI - my suggestion.

I hve a vB4 install with sphinx search and the rebuild by CLI take mere seconds, I believe that ES could achieve similar performance

 

[Naz]

Yes, this will be nice. How fast was the ES rebuild?

 

[Deriel]

In fact the rebuild of the Search was very fast - minutes for 3M posts (using 5k posts each run). My problem really is the Rebuild Users taking about 13h

 

[digitalpoint]

Elastic Search will never index in bulk as fast as Sphinx (unless ES makes major changes to how it works). Sphinx just talks directly to MySQL to pull records in bulk... Elastic Search goes through the application, so there's a decent overhead to having that added layer.

That being said, I did make a CLI-based search reindexer... but it's a little complicated for normal installs because it indexes in parallel across multiple servers. For example, if you have 8 servers running ES, indexing will be roughly 8x faster since each server is doing part in parallel. But for a single server setup, it wouldn't really have any advantage anyway...

 

[Deriel]

In fact my problem was the Rebuild Users - I was unaware of that differece at the time. My Rebuild Users is taking about 18 hours (in my migration testes)