Known Bots

Known Bots 6.0.3

No permission to download
  • new Cubot phone false positives:
    • cubot cheetah 2
  • weekly new bots:
    • better uptime bot
    • btcrawler
    • crawler_eb_germany_2.0
    • dle_spider.exe
    • dyno mapper crawler
    • flockbrain robot
    • infoobot
    • microadbot
    • nesotebot
    • reachabilitycheckbot
    • se ranking gentle bot
    • seobilitybot
    • siteguru linkchecker
    • statonlinerubot
    • tombot
    • viulinkcrawler
    • webgains-bot'
    • _zbot
  • new Cubot phone false positives:
    • cubot magic
    • cubot_manito
    • cubot_power
  • simplify seokicks-robot match to just seokicks to catch new bot user agent
  • weekly new bots:
    • applenewsbot
    • bl.uk_lddc_bot
    • dcrawl
    • dy robot
    • ezlynx
    • fast-webcrawler
    • gdark-spider
    • gethpinfo.com-bot
    • gowikibot
    • image size by siteimprove.com
    • linkcheck by siteimprove.com
    • loaderio;verification-bot
    • pingdompagespeed
    • pmoz.info odp link checker
    • refindbot
    • seebot.org
    • siteanalyzerbot
    • sitecheck-sitecrawl
    • sottopop
    • superbot
    • tineye-bot
    • webliobot
  • update: email subject line now includes addon version
  • feature: now optionally uses Monolog Logging Service addon for logging info about sent emails
  • fixed missing mailto: in some email links
  • new Cubot phone false positives:
    • cubot h3
    • cubot_note_s build/lmy47i
  • weekly new bots:
    • 3dd trunk
    • 3w24bot
    • auto spider
    • aylien
    • bidswitchbot
    • bnf.fr_bot
    • bublupbot
    • checkmarknetwork
    • chimebot
    • cloudservermarketspider
    • curious george
    • diffbot
    • everyfeed-spider
    • ffzbot
    • finditanswersbot
    • fyrebot
    • gigabot
    • graydon bot
    • hrankbot
    • huaweiwebcatbot
    • hypestat
    • hyscore
    • implisensebot
    • jasper's lil' bot'
    • jetslide
    • jpg-newsbot
    • konturbot
    • letsearchbot
    • looid.com crawler
    • magibot
    • mauibot
    • mindupbot
    • miralinks robot
    • onalyticabot
    • online-webceo-bot
    • queryseekerspider
    • randomsurfer
    • rankurbot
    • researchbot
    • safednsbot
    • serptimizerbot
    • sitecheckerbotcrawler
    • somdsearchbot
    • ssblog rsscrawler
    • sserobots
    • statdom.ru
    • synthesio crawler
    • toutiaospider
    • turnitinbot
    • website-audit.be crawler
    • who.is bot
    • wikido
    • wlc pywikibot
    • x28-job-bot
    • xyz spider
    • y!j-asr
    • yetibot
    • zombiebot
v3.3.0 updates:
  • added a phrase for email subject instead of hardcoding the string
  • removed Dalvik from list of bots - it's an Android browser
  • fixed missing user agent string for rssingbot
  • new Cubot phone false positives
    • cubot echo
    • cubot_j3
    • cubot king kong
    • cubot max
    • cubot note plus
    • cubot_note_s
    • cubot_note_s build/mra58k
    • cubot_nova
    • cubot r9
    • cubot_x18_plus
  • lots of new bots
    • adstxtlab.com
    • anderspinkbot
    • arquivo-web-crawler
    • barkrowler
    • botw spider
    • buzzbot
    • centro ads.txt crawler
    • cispa webcrawler
    • claritybot
    • cognitiveseo.com
    • crowdtanglebot
    • dingtalkbot-linkservice
    • elmer, the thinglink imagebot
    • elisabot
    • grfzbot
    • hubspot crawler
    • hubspot url validation check
    • icc-crawler
    • jobboersebot
    • kingbot
    • knot group
    • lawinsiderbot
    • makemoneyteamworkbot
    • mastodon
    • maxpointcrawler
    • mediapartners-google
    • mediumbot-metatagfetcher
    • metajobbot
    • msiecrawler
    • neticle crawler
    • netseer crawler
    • nextcloud server crawler
    • ninjbot
    • pagething.com
    • pagepeeker
    • pandalytics
    • parsijoobot
    • pimeyes.com
    • popscreen bot
    • pulno
    • pulsepoint-ads.txt-crawler
    • ravencrawler
    • redditbot
    • rely bot
    • rssmicro.com
    • sbooksnet
    • scanmine newsspider
    • seo-audit-check-bot
    • seoclaritycrawl
    • sitelockspider
    • slack-imgproxy
    • slackbot-linkexpanding
    • snappreviewbot
    • spiderling
    • squidbot
    • stormcrawler
    • superfeedr bot
    • superpagesbot
    • tmmbot
    • trendkite-akashic-crawler
    • ucmore crawler app
    • vebidoobot
    • yellowbrandprotectionbot
    • vkrobot
    • voilabot
    • wiederfreibot
    • yacybot
    • zumbot
Lots of new bots
  • 360spider
  • adsbot
  • adstxtcrawler
  • awariorssbot
  • bitlybot
  • boardreader
  • ccbot
  • cincraw
  • clickagy intelligence bot
  • crawlson
  • crsspxlbot
  • datagnionbot
  • deskyobot
  • df bot
  • dnsresearchbot
  • duckduckgo favicons bot
  • eyeotabot
  • feedlybot
  • foocrawlerbot
  • germcrawler
  • google adwords instant
  • gumgum bot
  • hatena
  • hetrixtools
  • hubpages
  • ias_crawler
  • internet structure research project bot
  • jugendschutzprogramm-crawler
  • krzana bot
  • lightspeedsystemcrawler
  • linespider
  • livelapbot
  • lufsbot
  • moatbot
  • netestate ne crawler
  • netpeakchekerbot
  • netvibes
  • nimbostratus bot
  • nixstatsbot
  • obot
  • odklbot
  • paperlibot
  • pilicanbot
  • pleskbot
  • politecrawl
  • pubmatic crawler bot
  • rogerbot
  • scooperbot
  • screaming frog seo spider
  • semantic-visions.com
  • semanticbot
  • seolizer
  • sirdatabot
  • surdotlybot
  • tapatalk cloudsearch platform
  • tpradstxtcrawler
  • velenpublicwebcrawler
  • yisouspider
  • wp.com feedbot
  • zoombot
Also found another set of false-positives:
  • Cubot mobile phones added to false positive list (Logicom BOT phones are already on the list)
v3.0.0 update (unreleased)

Major new feature: add generic bot detection

User Agent strings are scanned for the keywords "bot", "crawl", or "spider" - any User Agents not already detected as a bot which contain one of these strings are stored in the cache and made visible through the admin UI, with the option to have this information emailed on a weekly basis.

new bots: AccompanyBot; PostmanRuntime

v3.1.0 update (took less than 5 minutes after installing it on several of my production forums to identify new bots!!)

new bots: amazonbot; petalbot; slackbot
Lots of new bots, including UptimeRobot suggested by @Nirjonadda. See FAQ for complete list of bots now detected.

New functionality: tool to test bot detection - paste in a user agent string and the tool will tell you if it detects a bot
New bots:
  • TelegramBot (thanks @alsoGAMER )
  • InternetNZ Webscan
  • Microsoft WinHttp
  • Microsoft Office (Excel / Word / etc)
  • AspiegelBot (thanks @VersoBit )
Added an option to disable the display of robot statistics in sidebar widgets (which were added in v2.3.0 of this addon)
New bots:
  • 7Siters
  • AdScanner
  • BrandVerity
  • DuckDuckBot
  • e.ventures Investment Crawler
  • Exabot
  • HTTrack
  • Just-Crawling
  • Pinterestbot
  • Re-re Studio
  • SemanticScholarBot
  • Symfony BrowserKit
  • TraceMyFile
  • Ubermetrics
  • Voyager Bot
  • Linux Wget
Back
Top Bottom