Known Bots

Known Bots 3.20.0

No permission to download
Compatible XF 2.x versions
  1. 2.0
  2. 2.1
  3. 2.2
License
MIT
Visible branding
No
This XenForo 2.x addon adds additional definitions for bot detection in sessions.

This allows more accurate indication of the number of users vs robots online at any given time.

Requirements

This addon requires PHP 7.0.0 or higher and works on XenForo 2.x

Usage

When you look at Current Visitors, you'll see additional robots identified - also look at the "Robots" list on that page http://www.example.com/community/online/?type=robot

1569992317809.png



We also add the current robot count to the Members online widget and the Online statistics widget (from the current visitor page). This can be disabled via the options.

1574032520691.png


1574032528995.png


There is a tool in the admin area for displaying a complete list of currently known bots:

1589696038525.png



... and another tool which lets you test a user agent string to see if it would be detected as a bot or not:

1589696060644.png


v3 adds a new feature: generic bot detection

User Agent strings are scanned for the keywords "bot", "crawl", or "spider" - any User Agents not already detected as a bot which contain one of these strings are stored in the cache and made visible through the admin UI, with the option to have this information emailed on a weekly basis (not enabled by default).

The default email address to send new bots to will be delivered to me - however, the admin is free to have this information sent to any address they choose and can manually forward the detected bots to me.

To turn on automatic emails of detected user agents, check the Email New Bots option and enter an email (or leave it at the default setting).

1598534308662.png


There is an admin UI tool to display recently detected bots with a button to send the bot list email immediately:

1598534671528.png


Privacy Statement

v3 adds new functionality to email the list of new bots detected automatically to an email address configurable by the admin.

The default and recommended value is to email knownbots at hampel.io

Emails sent to this address will only be used for the purposes of identifying new bots to add to this addon.

Email addresses will NOT be sold or added to any marketing lists - not even ours.

If there are issues detected from the emails you are sending us, we may reply to establish communication - but that will be on a case-by-case basis and only for the purposes of troubleshooting the operation of this addon.

You can check the information contained in the emails by changing the address temporarily to your own so that emails go to you. If you still want us to process these emails - please feel free to forward them to the above address.

Other than a list of user agent strings, the only information contained in the email will be those automatically added to the email header by the forum mailer and SMTP servers.

Logging

v3.4 adds new functionality to optionally integrate with the Monolog Logging Service addon to log information about detected bot emails being sent. You simply need to install and configure the Monolog addon and a log file will be created with information every time a logging email is sent (minimum log level should be set to Info for logs to appear).
Related resources
Monolog Logging Service - optional integration for logging info about detected bot emails being sent
Author
Sim
Downloads
527
Views
11,285
First release
Last update
Rating
4.38 star(s) 8 ratings

More resources from Sim

Latest updates

  1. v3.20.0 June 2021 new bots

    new bots discovered in June 2021 btbot: BT Bot catchbot: Catchbot comodospider: Comodo SSL...
  2. v3.19.0 new bots

    new false positives: spider v9 phone new bots: 80legs.com: 80legs Crawler adssellerscrawler...
  3. v3.18.0 lots of new bots

    new false positives: cubot; j5 baiduboxapp new bots: 200pleasebot a8bot abilogicbot acoonbot...

Latest reviews

Adds the 'robots online' to the side widget, but not anywhere on the Members page. Pretty useless really.
Sim
Sim
The whole point of the addon is to identify a lot more Robots than the core product identifies - and these are shown on the Members page in the "Everyone" view and in the "Robots" view.
Really nice addon that removes "fake traffic" from showing up in the members online stats. Also well-made in the sense that it also finds new bots and can automatically report them back to the dev :) :+1:
This add-on will help me to know which bot has just visited my forum, its always interesting to know which bot is visiting.
This is an extremely useful addon for sorting out the bots you want crawling your site versus those that are just nuisance bots taking up bandwidth and giving nothing useful back. It's especially useful as a starting point if your server is being overrun by Chinese and Russian bot to the point where it is intermittently bringing down the server in a DDoS-like scenario (which happened with one forum I was brought in to repair).
A useful extension to XenForo's bot classification system to handle more bots that are seen in the wild
Great effort and I'm sure it's a well made addon, but the bots visiting my website are still coming up as guests after correct addon installation. It would be good if there was a way to easily add entries. For example, (ecs-114-119-162-196.compute.hwclouds-dns.com) is not showing up as a bot, but as a guest, so I assume the addon doesn't have this listed. I have 60 "guests" and 95% of them are bots.
Sim
Sim
This addon cannot and will not identify bots by IP address or hostname - it only uses the user agent. If you have identified bots which are masking their user agent, you should block them at the web server level.
A very simple plugin that does what it says on the tin. I recommend if you love having statistics, like me!
I think this is a very underrated addon. Every board has a lot of bots lurking in the background and this useful addon helps you to separate them with your normal visitors. Because it then shows the real amount of guests you have right now and not the fake number which comes from the bots. Thank you.
Top