I'm on their free plan and still fine tuning, seeing what works best for me. I was getting a lot of fake Googlebots, made some changes and haven't seen them for a while now.
Crazy amount of guests
- Thread starter Levina
- Start date
-
- Tags
- bot swarms guests
Cloudflare definitely has a learning curve. I now make a change and wait, monitor what it does before doing anything else. It seems things are set nicely now though and traffic is down to between 300 and 500 guests so I'm happy even though not all are genuine guests.
I've set custom rules for countries, some ASN's, and some very specific bots (Sim's add-on was quite helpful with this!) and it seems all that's left is residential proxies. Maybe it would be better to turn on Bot Fight Mode instead of using custom rules, but on the fence about it as I read conflicting reports about it.
I set it up yesterday and it went down from 85,000 guests to about 12,000 so whatever I did worked, but now it's back up to 85,000 again
I blocked Vietnam, but Vietnam is still showing up as the worst defender in Analytics. Also, when I select country to block, there's a drop-down - I'm assuming for country I select is equal to. Is this correct.
They found a chink in my armor and exploited the hell out of it. It's since reinforced.
The blast was 4 minutes long and a peak of 1600 tcp/ip ports were occupied. Glad i set a max limit of 2048 and also have 128 spare apache processes. So the site only hiccupped for one second.
The design of my system has a day of work left. So we're going to give XF Bot Guard a shot soon.
The blast was 4 minutes long and a peak of 1600 tcp/ip ports were occupied. Glad i set a max limit of 2048 and also have 128 spare apache processes. So the site only hiccupped for one second.
The design of my system has a day of work left. So we're going to give XF Bot Guard a shot soon.
Yes:
Country - Equals - Vietnam
Then add more with OR.
I have a long list of countries blocked and as soon as it was activated my numbers plummeted from over 30k to 800 or so. Other rules brought it down even more. My visitor count is pretty steady now around 350. There's still residential proxies among them, but I'm pretty happy.
it is hard to keep a track on this thread lol. but if you are using the digitalpoint addon, he has country blocking baked in which is slightly easier to use from xenforo backend itself. just a fyi! i removed my manual country block list rule recently and started using the addon's.
Attachments
Have you set 'under attack mode' in cloudflare during that time?
Some analysis from the attack:
It looked like a 50:50 mixture of residential proxies ( USA based ) and datacenters. Datapacket being very prominent
I would expect something that could reference a list of datacenters, for example, if it used proxycheck.io, the elimination rate for me would have been decent.
Pepe, how many posts does your site have? i have 94k posts. So there's a lot for crawlers to hit..
By looking at the ip addresses
CF analytics? It should... as blocked. CF analytics shows everything hitting CF, your server analytics shows what hits your server. If they aren't getting through CF, then your server is good. If they are, then is your server locked to CF only IP's? Otherwise they bypass CF as your origin IP is easy to obtain.
Mine are slowly coming back now I have removed all blocks. Experiment. Taken them a month to increase only a few hundred thousand... see what the following months bring.
Absolutely insane compared to my numbers. We have ~94k posts.
What was your baseline of unique visitors approximately before the experiment?
What was your baseline of unique visitors approximately before the experiment?
Last edited:
Your forum shows:
Sorry, flipped the figures around :O
On my one client that i'm using cloudflare on, i decided to give a look at how it's doing.
We have a mostly stock cloudflare configuration defending a magento store. A few ASN bans. A few geographic regions get challenged. Some FAANG scrapers are allowed, but a few abusive ones are banned.
Looks like the proxies are walking right through it quite a few times.
We had to turn on 'under attack' mode today. We have to do this once or twice a month. At this point, we are going to automate it.
What's unfortunate is the bandwidth consumption number is ~20% higher than when i had the nicely catered fail2ban setup. That might be a variation for this month. But i would expect cloudflare to outperform my rate limiting setup.
The only reason we have it is for 'under attack mode' when the residential proxies come in heavy and fast. Otherwise after a few months of using it, i'm not too impressed.
We have a mostly stock cloudflare configuration defending a magento store. A few ASN bans. A few geographic regions get challenged. Some FAANG scrapers are allowed, but a few abusive ones are banned.
Looks like the proxies are walking right through it quite a few times.
We had to turn on 'under attack' mode today. We have to do this once or twice a month. At this point, we are going to automate it.
What's unfortunate is the bandwidth consumption number is ~20% higher than when i had the nicely catered fail2ban setup. That might be a variation for this month. But i would expect cloudflare to outperform my rate limiting setup.
The only reason we have it is for 'under attack mode' when the residential proxies come in heavy and fast. Otherwise after a few months of using it, i'm not too impressed.
We've been running fine for the past 2-3 weeks using Clouflare and various rules in place.
Then at 4:00am BST today we got attacked by tens of thousand of 'Guests' resulting in page loading times to increase and eventually 'UAM' being implemented.
Crazy, but when it does happen it always seems to start around 4:00am BST
Then at 4:00am BST today we got attacked by tens of thousand of 'Guests' resulting in page loading times to increase and eventually 'UAM' being implemented.
Crazy, but when it does happen it always seems to start around 4:00am BST
Ultra Administrator Mode?
User Alien Mixer?
Us Alone Minimizer?
Urban Auto Moderation?
Please: Do not use uncommon abbreviations and mention at least the tool, you are talking about. This thread is not about a single tool and many readers of this thread are not familiar with every tool and it's details, especially, if they are not using it themselves. I assume, after some guessing, you may be talking about "under attack mode" in Cloudflare?
I have left Bot Fight Mode on and haven't noticed any complaints about it. I leave it on despite having a lot of custom rules. Because we want no AI scrapers (and do not allow AI responses in our forums), I have also enabled AI Labyrinth and Block AI Bots. I refuse to feed the oligarchs.
When we had a wave of attacks, I have activated Under Attack Mode, but only temporarily to knock the bots back a bit.
