Known Bots

Known Bots 3.8.0

No permission to download

Sim

Well-known member
Sim submitted a new resource:

Known Bots - Adds additional definitions for bot detection in sessions

This XenForo 2.0 addon adds additional definitions for bot detection in sessions.

Requirements

This addon requires PHP 5.4 or higher and only works on XenForo 2.0.x

Usage

When you look at Current Visitors, you'll see additional robots identified - also look at the "Robots" list on that page http://www.example.com/community/online/?type=robot

Read more about this resource...
 

Brad Padgett

Well-known member
Great work. This seems like a great add-on. If this allows robots to be defined in members online that's a great feature missing from Xenforo 1. I may use this at some point. Anyways good job!
 

Sim

Well-known member
I checked the Online members widget, and it filters the results to exclude bots - which uses the session information, which is what checks the user agent against the robot list.

So the Online members widget doesn't actually show robots, but the information is does show will exclude robots.
 

Sim

Well-known member
Still does not understanding what does this add-on, do you have any preview?

I don't have a production XF2 forum operating yet, so I don't have any real world data to show screenshots for. I've been running an XF1.5 version of this for a few years now though.

If you look at the "Current Visitors" list on the Members tab of your XF2 based forum, you'll see a Robots tab (this information is also visible in the Everyone tab, mixed in with the Members and Guests).

It looks something like this:

1528816181008.png

Those robots are detected by the forum software by analysing the User Agent data of the incoming request.

This addon expands the detection capabilities to include a long list of other robots (ie not human visitors).

I might add a list of currently supported bots to the FAQ section.
 
  • Like
Reactions: Xon

Claverhouse

Active member
Will these appear in the Online members widget as well?


That would be great, I found this whilst looking for a replacement for the old Wolfshead spider list, which showed individual bots. I don't have strong feelings on bots, for or against; but I object to their presence counting as guests, giving a false sense of how popular a small forum may be.
 

DragonByte Tech

Well-known member
DB Security took (with permission) a spider XML file made for vBulletin and converted it into a generic array which extends \XF\Data\Robots::getRobotUserAgents() / \XF\Data\Robots::getRobotList() and adds their identifiers.

Whether the list matches this mod's, I have no idea, the conversion was done before the initial XF launch of the Security mod :p


Fillip
 

Sim

Well-known member
Whether the list matches this mod's, I have no idea

I doubt that it would match - I did not use any data from vBulletin, but merely extended the XF list with my own list gathered from my website logs - it is by no means exhaustive and no doubt misses some of the bots you've got while I'm sure there is also some overlap.
 

Vekseid

Active member
You're clobbering the original methods with this. You should build independent arrays and return an array_merge instead.
 

Sim

Well-known member
You're clobbering the original methods with this. You should build independent arrays and return an array_merge instead.

Meh, the list of bots they cover in the core hasn't changed in quite a few years - the oldest version of XF I still have a copy of is XF 1.2.1 from 2013 and it had exactly the same list of bots as is in XF2.

It's trivial to add new bots to my list and push a new version of the addon if the team at XenForo choose to add extras and it's not going to break anything if they do add something anyway.
 

Sim

Well-known member
Sim updated Known Bots with a new update entry:

Re-release

Same version, new addon_id.

If you already have v2.0.0 installed, there is no need to upgrade. There is no new functionality in v2.0.0a. New installations should use this v2.0.0a version to ensure compatibility with future releases.

Due to the change in addon_id, upgrading from v2.0.0 to v2.0.0a is not supported - but there is no data stored with this addon, so simply uninstalling v2.0.0 before installing v2.0.0a will be sufficient.

Read the rest of this update entry...
 

Nerdface

Well-known member
DB Security took (with permission) a spider XML file made for vBulletin and converted it into a generic array which extends \XF\Data\Robots::getRobotUserAgents() / \XF\Data\Robots::getRobotList() and adds their identifiers.

Whether the list matches this mod's, I have no idea, the conversion was done before the initial XF launch of the Security mod :p


Fillip
Can I be cheeky and ask whether there's any chance of sharing said list? ;) No worries if not.
 
Top