XF 1.5 my xf_search table is massive all of a sudden, rebuild doesn't help

sross

Active member
Hi, not sure what is going on here. I noticed a backup taking too long to find my xf_search table had blown out to 85 GB. My database backups are usually 1.1 GB. Forum has 2.4mil posts. I am running elastisearch. I emptied the table and re-ran the indexing. It's been going for hours and table is now 25 GB and rising. Not sure what to do. Thanks for any help.
 

sross

Active member
in case this helps anyone with similar issue. I might be in the clear.. I updated my robots.txt and google claims to follow this. I then went to search console at google and re-submitted my updated robots.txt which included the disallow for /find-new/

I then had google test the url and the bot claimed it was blocked. Will see if this sorts it out in the next day or so

Screen Shot 2022-01-13 at 11.44.27 AM.png

*update

so far no change, so although google claims their bot will honour the robots.txt doesn't seem to unless it takes a while to kick in. I'll see if I can find any support at google regarding this

*update

you can see where this started all of a sudden

Screen Shot 2022-01-13 at 2.30.15 PM.png

*one more update

I might be in the clear now. I discovered several url variations in my google search console and those did not contain the updated robots.txt blocking /find-new/ block. I updated that and the pounding seems to have stopped. My server load has dropped from 15.0 to 0.70 still monitoring, also xf_search is not growing and lurking around 15mb
 
Last edited:
Top