It's sort of up to you, but XF Enhanced Search can provide better results through relevancy ordering, stemming, fewer stop words and no minimum word length limit. It's certainly something you could add at some later point if you think the built-in search isn't up to scratch.
I only use Elastic Search on my big board but not on other sites. as the ES integration does not allow you to use the stock Elastic Search functionality, nor does it offer any additional functionality. Asides from performance and somewhat better search results, its pretty bare bones.
I hope that in the future the Enhanced Search addon will be more fleshed out so that it would offer compelling features to use it on all my sites.