How To Add User Agents to Spider's Listing?

library/XenForo/Session.php

Code:
	/**
	 * Known robot user agent substrings
	 *
	 * @var array
	 */
	public static $knownRobots = array
	(
		'avsearch',
		'baiduspider',
		'bingbot',
		'crawler',
		'facebookexternalhit',
		'feedfetcher-google',
		'feedzirra',
		'googlebot',
		'kscrawler',
		'magpie-crawler',
		'nutch',
		'php/',
		'scooter',
		'scoutjet',
		'sogou web spider',
		'twitterbot',
		'xenforo signature generator',
		'yahoo! slurp',
		'yandexbot',
		'zend_http_client',
	);

It checks the user agent for those strings.
 
library/XenForo/Session.php

Code:
/**
* Known robot user agent substrings
*
* @var array
*/
public static $knownRobots = array
(
'avsearch',
'baiduspider',
'bingbot',
'crawler',
'facebookexternalhit',
'feedfetcher-google',
'feedzirra',
'googlebot',
'kscrawler',
'magpie-crawler',
'nutch',
'php/',
'scooter',
'scoutjet',
'sogou web spider',
'twitterbot',
'xenforo signature generator',
'yahoo! slurp',
'yandexbot',
'zend_http_client',
);

It checks the user agent for those strings.
Thanks but how do I add the spiders from this list? http://www.wolfshead-solutions.com/spiders-list

Do I just type a new line and put the name of the user agent between quotation?
 
Rich (BB code):
	/**
	 * Known robot user agent substrings
	 *
	 * @var array
	 */
	public static $knownRobots = array
	(
		'avsearch',
		'baiduspider',
		'bingbot',
		'crawler',
		'facebookexternalhit',
		'feedfetcher-google',
		'feedzirra',
		'googlebot',
		'kscrawler',
		'magpie-crawler',
		'nutch',
		'php/',
		'scooter',
		'scoutjet',
		'sogou web spider',
		'twitterbot',
		'xenforo signature generator',
		'yahoo! slurp',
		'yandexbot',
		'zend_http_client',
		'new',
		'user',
		'agents',
	);
 
I'm sorry I'm a little slow Jake but I need clarification again after looking at this file of 620 spiders from Wolfhead solutions. In the excerpt below I see two names for one spider. Do I enter the name of the 'spider ident' line as the user agent or do I enter the 'name' as the user agent?

Code:
    <spider ident="BigmirSpider">
        <name>Bigmir</name>
        <info>http://www.bigmir.net/</info>
    </spider>
    <spider ident="bingbot">
        <name>Bing</name>
        <info>http://www.bing.com/bingbot.htm</info>

I'm using this example because you have bingbot as the name of the user agent in your example but in his file the name says Bing.
 
I've just added some spiders and then ran the "Health Check" in Admin CP and it fails because the file has changed. Might be a good idea to not include that file in health checking, or whatever so everything still passes OK if adding extra spiders many people will do?
 
Top Bottom