XF 2.2 Battling Bytespider - discouragement mode not working (it did at first)

[FoP]

A month ago we were overrun by the Bytespider bot and I added it to the robots.txt file. It worked for a month but a few days ago they came back in full force. We are on cloud hosting and as such have no access to the .htaccess file, so I decided to put the entire IP range the robot is using in discouragement mode, redirecting them 100% to some search engine. That worked for a day and a half. Now they're back and I don't know how this is possible.

Looking at the log, it says the discouragement mode for the IP range was last triggered yesterday at 15:15 hrs. But the Robots tab in Current Visitors is filling up with them as I type these words. So why is the discouragement mode not working? It did at first, but it's like they found a workaround.

The good thing is every bot has the "viewing an error" sign to its name, so I am assuming they're not viewing any content at all, but I really want them gone altogether.

 

[ekool]

You could block them pretty easily with Cloudflare... does your shared hosting allow you to block IP's at all? Null route? You could just ban the IP address range in Xenforo as well.

 

[Chromaniac]

yeah unless xf cloud offer more protection or blocks them at source which i think they mentioned they are considering, cloudflare or a similar product would provide you with some form of control for these kind of purposes.

 

[FoP]

You could just ban the IP address range in Xenforo as well.

I did that yesterday and that's maybe the reason they are viewing errors? Not sure. But I just checked and the banned IP range was triggered "a moment ago," so that's working. Why the discouragement mode isn't triggered anymore, I don't know.

unless xf cloud offer more protection or blocks them at source which i think they mentioned they are considering

I'm still hoping for this, but there's been no word on it since @Chris D mentioned it a while ago.

If not, yes, Cloudflare would probably be the best option.

 

[Chromaniac]

Why the discouragement mode isn't triggered anymore, I don't know.

well discouragement does not send them outside the website or stop them from accessing the website. they would still be on the website. accessing content (or blank page) with delay and other tactics.

Discouraging aims to make using the site slow and painful for a problematic user, such that they eventually give up and leave. According to the system configuration, each time a user subject to discouragement attempts to load a page, there is a chance that it will fail to load, a change that it will result in a blank page, a change that it will take a very long time to complete etc.

Discouraging users | Manual | XenForo

End-user documentation for XenForo

xenforo.com

I had completely forgotten Banned IP addresses existed in XenForo. You should be able to use this! As @ekool mentioned!

Banning users | Manual | XenForo

End-user documentation for XenForo

xenforo.com

 

[FoP]

well discouragement does not send them outside the website

But doesn't the redirect send them to the URL you fill in? I set that at 100%, so logic dictates that all traffic from the IP range should be redirected to the external site.

But even if this doesn't happen, the discouragement mode should still be triggered when the IP range visits our forum and it isn't.

I had completely forgotten Banned IP addresses existed in XenForo. You should be able to use this! As @ekool mentioned!

Yes, I set that yesterday and it's working but it doesn't stop them from visiting. I would really like them gone.

 

[Chris D]

There’s not a clear catch all solution at this point. I understand Cloudflare can help and using their services to keep your IP protected and a configurable WAF is recommended anyway so I’d say that’s probably the most direct workaround at this point.

But when you say “overrun” what is your main concern exactly? Is the forum not performing well? Or are you concerned about page view counts?

 

[FoP]

well discouragement does not send them outside the website or stop them from accessing the website. they would still be on the website. accessing content (or blank page) with delay and other tactics.

But they're listed as viewing threads and images, albeit getting a viewing error for all.

What I don't understand is why the Discouraged IP mode is not triggered by the bot. No matter what, that should happen, no?

But when you say “overrun” what is your main concern exactly? Is the forum not performing well? Or are you concerned about page view counts?

The forum is performing well. I just don't like Chinese bots roaming our members' data and images. It just bugs me as there is really no good reason for them to visit us, other than stealing our data. And they ignore the robots.txt file, which makes it a bad bot, to say the least.

I wish we had the ability to block an entire country. That would take care of some so called "guests" as well.

Oh well, Cloudflare it is then.

 

[Chris D]

The forum is performing well. I just don't like Chinese bots roaming our members' data and images. It just bugs me as there is really no good reason for them to visit us, other than stealing our data. And they ignore the robots.txt file, which makes it a bad bot, to say the least.

Fair enough. Just so people are aware, bot traffic is filtered from page view allowances (as best as possible) so that would be taken into account.