[DigitalPoint] App for Cloudflare® 1.9.1.1

[digitalpoint]

There is no "contains" operator for user agent rules. If you want to use "contains" for user agents, you can use a firewall rule (more flexible, but also more complicated):

http://dash.cloudflare.com/?to=/:account/:zone/security/waf/custom-rules/new

The rule someone posted above is mostly right... general idea anyway (except ByteDanceSpider isn't a user agent that would match).



Again... not a specific user agent rule, but it also doesn't need to match exactly.

 

[Alvin63]

Thanks. Just checking I've been looking in the right place in the app. I was under Firewall - User agent - to set up a rule.

In settings there is a Bots section - should I be using that instead? Although don't want to block all bots obviously.

 

[digitalpoint]

Thanks. Just checking I've been looking in the right place in the app. I was under Firewall - User agent - to set up a rule.

In settings there is a Bots section - should I be using that instead? Although don't want to block all bots obviously.

User agent rules are exact match. The issue is you don't have access to your web server logs, so you don't actually know what the user agent is (exactly anyway). The link I gave you above will allow you to do user agent blocking with a contains operator.

 

[Alvin63]

There is no "contains" operator for user agent rules. If you want to use "contains" for user agents, you can use a firewall rule (more flexible, but also more complicated):

http://dash.cloudflare.com/?to=/:account/:zone/security/waf/custom-rules/new

The rule someone posted above is mostly right... general idea anyway (except ByteDanceSpider isn't a user agent that would match).

View attachment 323756

Again... not a specific user agent rule, but it also doesn't need to match exactly.

Thanks. That didn't seem to work either apparently. So I tried blocking Singapore. And it's still there! With Singapore next to the name. I think it might be bypassing Cloudflare.

 

[digitalpoint]

Thanks. That didn't seem to work either apparently. So I tried blocking Singapore. And it's still there! With Singapore next to the name. I think it might be bypassing Cloudflare.

It's certainly possible... but without access to web server logs, you are kind of flying blind.

 

[Alvin63]

Thanks. I'll give up for now. That would be concerning though if Bytedance knew the origin IP of the website. I suppose it could have obtained it before I set up the app to proxy unfurling .......... If it is bypassing Cloudflare though, then presumably the only option would be to block it in htaccess? Or range block the IP's in the server?

 

[digitalpoint]

It may not be that it tried to determine your server IP specifically, rather they could just be spidering random IPs.

If you have the ability to block IPs at the web server level, I'd block all IPs other than Cloudflare's because if you are proxying through Cloudflare, it should ONLY be Cloudflare IPs making connections to your server. You can find all Cloudflare's IP ranges here:

IP Ranges

This page is intended to be the definitive source of Cloudflare’s current IP ranges.

www.cloudflare.com

 

[Alvin63]

It may not be that tried to determine your server IP specifically, rather they could just be spidering random IPs.

If you have the ability to block IPs at the web server level, I'd block all IPs other than Cloudflare's because if you are proxying through Cloudflare, it should ONLY be Cloudflare IPs making connections to your server. You can find all Cloudflare's IP ranges here:

IP Ranges

This page is intended to be the definitive source of Cloudflare’s current IP ranges.

www.cloudflare.com

Thanks. I had intended to do that before, but it wasn't possible. The server won't let me whitelist the full IP ranges. Only all the individual Cloudflare IP's and I worked out there were probably thousands! Eg there is a list of 20 or so Cloudflare IP's but each of those is a range with many more included.

 

[digitalpoint]

Thanks. I had intended to do that before, but it wasn't possible. The server won't let me whitelist the full IP ranges. Only all the individual Cloudflare IP's and I worked out there were probably thousands! Eg there is a list of 20 or so Cloudflare IP's but each of those is a range with many more included.

Ya, that definitely won't work. There are 524,285 IPs in each /13 block alone.

Maybe time to think about better hosting? Seems like they aren't that great in all the ways.

 

[Alvin63]

Ya, that definitely won't work. There are 524,285 IPs in each /13 block alone.

Maybe time to think about better hosting? Seems like they aren't that great in all the ways.

I know I went round in circles with this before. The only other plan that would allow it is a VPS hosting plan and that's a bit beyond my technical skills.

 

[Alvin63]

I've sent a request to the server to block ByteDance and Bytespider at server level, which I believe some servers do. But they will probably ignore me

How about the AI Labyrinth option in the addon?

 

[digitalpoint]

I've sent a request to the server to block ByteDance and Bytespider at server level, which I believe some servers do. But they will probably ignore me

How about the AI Labyrinth option in the addon?

Well if they are hitting the server direct, none of the Cloudflare settings will do anything because the traffic isn’t passing through Cloudflare.