XF 2.2 Anyway of limiting bandwidth in order to prevent site scraping?

jromaine

jromaine

Active member
Hi guys,
I had a member recently use a "scraping tool" to essentially download over 300 Wistia videos, along with hundreds of resources and a lot of written content. While this was happening I could see their user profile "online" for a lengthy period, overnight infact which caught my attention. Especially given most users tend to drop off after a few minutes. Upon logging into the admin panel I could see a whole bunch of error messages. When I say a whole bunch I mean hundreds!

ErrorException: [E_NOTICE] Undefined offset: 1 src/addons/IntegratePro/Wistia/Wistia.php:31
Generated by: (username removed) Oct 31, 2022 at 12:58 AM

I wanted to ask....

Is there any way, I can force end of sessions for members that show high bandwidth usage or abnormal behaviour within Xenforo?

Even if I could have the system notify me via email so I could investigate.

Obviously I'd prefer this didn't happen again.

Thanks
 
Sort by date Sort by votes
Why not ban the user? This can be done via IP range, you can also set them as a discouraged user, also by username & IP range, so that they get slowdowns and other annoying occurrences and this may prevent them from scraping your content.
 
Upvote 0 Downvote
DarkGizmo said:
Why not ban the user?
Click to expand...
I think the OP wants to be sure this does not happen with other accounts again.

On the XF-side this is not easy to accomplish. You could use this addon (maybe there are other similar ones, this is the only one I am aware of):

xenforo.com

[WMTech] Time Spent Online

If you like this add-on, please >> rate it
xenforo.com xenforo.com

Then setup a user promotion, that adds a user-group with limited rights after some time spent online (I don't know how flexible this addon is regarding time spans and duration of online activity). To be clear: This addon does not measure bandwidth, but only time spent online.

Another idea: use this (free) addon by @Xon along with his Redis Cache addon:

xenforo.com

User Activity by Xon

Displays user activity below content. Supported content: Threads Conversations Reports NixFifty's Tickets NixFifty's Calendar It is recommended (but not required) that Redis Cache add-on be installed and configured as the cache provider...
xenforo.com xenforo.com

It won't help you out of the box, but with a custom addon on top of this you may be able to limit the requests and/or take actions.

But, ideally this need to be done on the server-level, I guess. Maybe Cloudflare can be helpful for this.
 
Upvote 0 Downvote
DarkGizmo said:
Why not ban the user? This can be done via IP range, you can also set them as a discouraged user, also by username & IP range, so that they get slowdowns and other annoying occurrences and this may prevent them from scraping your content.
Click to expand...

I've already banned them. I booted them immediately and cancelled their subscription.

Up until finding this out recently they've been a valued member, no issues at all.

I can't run around "banning" users unless I know they've actually done something wrong, which is exactly why I'm asking the question.

Is there any way to detect this type of behavior so I can intervene/investigate and action accordingly?
 
Upvote 0 Downvote
P.S. regarding clouldflare, look at this article:


Example:


previous-rate-limiting-create-rule.png
 
Upvote 0 Downvote
You must log in or register to reply here.
Top Bottom