Lack of interest Spiders XML?


Well-known member
I looked but didnt see one, But then im famous for overlooking things.. So is there a Spiders XML in the XF package? and if so how would one go about adding spiders to the list?
I know Dream (i think it was dream) had a modified spider xml for vB with hundreds of spiders listed.. Id like something like that again.


Well-known member
what's the status of this?
Are the robots hardcoded in xf 1.2?
is there a global database which sync feature or any other import feature?


Well-known member
Without seeing code, I know it was hardcoded into v1.1 somewhere. I know there was a thread about it not to long ago.

@xf_phantom: 1.1 has it in XenForo_Session::$knownRobots and XenForo_Session::isRobot().

Jake Bunce

XenForo moderator
Staff member

	 * Search engine domains (excluding TLD)
	 * @var array
	public static $searchDomains = array

	 * Known robot user agent substrings
	 * @var array
	public static $knownRobots = array
		'sogou web spider',
		'xenforo signature generator',
		'yahoo! slurp',
This can be changed with a file edit or an addon that changes these public static vars of XenForo_Session.


XenForo developer
Staff member
The code for the robots has changed, but it is roughly in the same location as what Jake has pointed out. The manner in which an add-on would modify it won't be the same as in 1.1.


Well-known member
Is the XF 1.2 spiderlist much expanded? The above is a very short list compared to the spiderlist that mosh made for vb or that bad behavior has.


Well-known member
IMO this is already available
We can add to $_knownRobots and $_robotMap via event listeners

correction: we can't, because they're not static
Last edited:


Well-known member
Not everyone is a developer.

I suspect the suggestion is to do with an easy way to create a list in the ACP.

and i was also wrong, we can't just add them..:/
We would have to overwrite the session class instead of just adding it to the array, because he's not static