XF 2.3 Sudden surge of traffic to the What's New / New posts pages
- Thread starter JoyFreak
- Start date
![[XB] Search Spaminator: Search Bot Protection, AI Scraper Blocking & Honeypot](/community/data/resource_icons/10/10953.jpg?1792894918){kind=icon}
Have you tried it ?
If you download the raw server logs and ask Claude to analyse them it may well show a large bot swarm of residential proxies using Chrome 151. That is what my swarm is. 99% of them were going direct to whats-new. 30% could solve cloudflare challenges. But setting up Cloudflare rules reduces it about 70%.
Even if you turn off Whats New for guests in Xenforo, the bots just go direct to the path.
If you set up a custom rule in Cloudflare and set it to "managed challenge" it will challenge the bots and stop about 70% of them. This rule whitelists logged in members and known bots like google so they don't get challenged.
(starts_with(http.request.uri.path, "/whats-new/") and not http.cookie contains "xf_user" and not cf.client.bot
Edit: I also add referer eq "" to my rule - meaning genuine guests don't get challenged as they have a referer usually by landing on a home page or from google. Whereas bots go direct so don't have a referer. Some bots fake a referer so it does let a few more of them through but still stops the bulk of them. The most virulent ones on my site didn't have a fake referer. Claude can also tell you which ones have a fake referer from the server logs as well.
I actually blocked them in the end with a block rule - but after being blocked for 12 days, they changed pathway to "goto" - so I set up a new rule for goto as well. Now I've set up a honeypot and that catches all of them but I did that in Cloudflare snippets which I think is only in the pro version.
Last edited:
Came up with it. Had it awhile.
Ironical isn't it - someone else on here gave me the tip and it's very useful. Claude can tell you which type of browsers in which swarm going to which path and you can set up the rules accordingly. They are 99.9% all using desktop browsers as well so you can tailor a rule to only challenge desktops using a particular browser and allow mobiles past. To avoid challenging as many real users as possible.
Ironical because the botmaster behind these swarms maybe used AI to set them up as well. These bots know all the Xenforo pathways.
Last edited:
Could you do one for Whats New as well?!!
Not everyone uses Cloudflare.
It challenges ALL automated attempts to execute ANY search, configurable.Could you do one for Whats New as well?!!
Yeah these scrapers aren't programmed to submit forms. Makes them easy to spot. Achilles.I haven't tried the one for search - but used the registration one
Last edited:
Similar threads
- Question
