in case this helps anyone with similar issue. I might be in the clear.. I updated my robots.txt and google claims to follow this. I then went to search console at google and re-submitted my updated robots.txt which included the disallow for /find-new/
I then had google test the url and the bot claimed it was blocked. Will see if this sorts it out in the next day or so
*update
so far no change, so although google claims their bot will honour the robots.txt doesn't seem to unless it takes a while to kick in. I'll see if I can find any support at google regarding this
*update
you can see where this started all of a sudden
*one more update
I might be in the clear now. I discovered several url variations in my google search console and those did not contain the updated robots.txt blocking /find-new/ block. I updated that and the pounding seems to have stopped. My server load has dropped from 15.0 to 0.70 still monitoring, also xf_search is not growing and lurking around 15mb