[TAC] Bot Arrestor 2.0.12

If you like this add-on, please >>rate it<<

Included in the [TAC] Total Anti-Spam Collection.

Description
Bots that assail your forum for malicious purposes, repeatedly interact with your forum in a sustained and resource draining manner. This could be as they scrape your content, or try to register so they can spam. Not only does your forum suffer when they succeed in these acts, but their sustained attempts slow down the forum for everyone else.

[TAC] Bot Arrestor mitigates this, sidelining the bots so they cannot continue to assail the forum.

Features

• Lowers bandwidth & database resource usage from spam bots, scrapers, and non-distributed denial-of-service attacks.
• Allows spiders/crawlers like Google to continue unaffected.
• Dynamically synchronises the .htaccess file, for a truly ‘zero query’ method of stopping bots from hammering your forum.
• Uses JavaScript detection to remove users from the cache (avoiding false positives).
• Ignores logged in users.

This plugin primarily targets spam bots or scrapers with high resource usage. When this plugin is used in combination with [TAC] Fool Bot Honey Pot, a large percentage of spam bots are detected and cached. [TAC] Fool Bot Honey Pot detects bots that attempt to register, and then caches them; and [TAC] Bot Arrestor detects bots that attempt to quickly login / register / scrape pages over and over, and then caches them. Once cached, these spam bots use minimized server resources.

By default, the ACP options for Bot Arrestor are set up so that humans will rarely ever see the warning page (if at all, unless they are malicious), but it will still catch spam bots that would have used significant resources.
Note: this is not a preventive measure for DDOS attacks (those from many thousands of IP addresses usually from botnets). Those are best mitigated with a commercial DDOS protection service such as CloudFlare.

How it works
If the user hits 6 pages or more within 7 seconds (something a human wouldn’t do), a friendly user message is displayed to the user. This friendly message then counts down and redirects them to the original page. If they continue to hit more pages after seeing the message (bots will, humans shouldn't), and they hit 8 pages or more within 7 seconds, they are locked out of the site and their IP is cached. From then onwards, that IP will only see a 401 Unauthorised page (and only take up 1 query instead of 15 to 25 queries).

Screenshots

Friendly User Dos Message if the user hits 6 pages or more within 7 seconds:


After further attempts and hitting 8 pages of more within 7 seconds, this message is all they will see site wide (their IP is cached):


Spiders/Crawlers exclusion
Spiders/Crawlers (such as Google) can hit many pages quite quickly, however this type of bot is permitted by Bot Arrestor. Bot Arrestor uses the XenForo core methods to avoid detecting these types of bots and also looks at the User Agent. Spam Bots will almost always disguise themselves as normal browser users, whereas spiders/crawlers will always exposes their selves with the user_agent (user_agent is always logged to confirm spiders have not been stopped). If the user agent does not look like a browser, the Bot Arrestor ignores them (since it could be an unknown spider/crawler). The user_agent of each arrested bot is always shown in the logs.

Logs:


ACP options:


Logs are automatically cleaned up weekly, so no more than 3 months of logs are stored (preventing the logs from building up).

Install & Upgrade

• Installation instructions
• The options have sensible defaults, but you can set them in the administration control panel: ACP -> Home -> Options -> Bot Arrestor

Warning: If you decide to test this plugin on your own forum, and if you decide to refresh the page more times after seeing the warning message, you will be locked out of your site (including the ACP area). In such cases, you can turn off the Bot Arrestor cache by turning on debug mode, then login to your ACP and remove your IP from the cache, and you can then turn debug mode back off.