Known Bots

Known Bots 6.1.0

No permission to download
Overview Updates (36) Reviews (9) History Discussion
New bots detected [3.16.0]
  • mozilla/5.0 (linux; android 7.1; spider v7) applewebkit/537.36 (khtml, like gecko) chrome/86.0.4240.198 safari/537.36 opr/61.2.3076.56749
 
Nirjonadda said:
New bots detected [3.16.0]
  • mozilla/5.0 (linux; android 7.1; spider v7) applewebkit/537.36 (khtml, like gecko) chrome/86.0.4240.198 safari/537.36 opr/61.2.3076.56749
Click to expand...

This one is actually a false-positive! The MyCell Spider v7 phone is from Balgladesh I believe.

It's still useful - I'll add it to the false positive list - thanks.
 
New bots detected [3.16.0]

  • mozilla/5.0 (linux; android 5.1; spider v7 build/lmy47i; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/43.0.2357.121 mobile safari/537.36
  • mozilla/5.0 (compatible; ampxfbot/1.0; +http://ampxf.com/bot)
One from @mazzly addon. Identified as: Mozilla/5.0 (compatible; AMPXFBot/1.0; +http://ampxf.com/bot)
 
Last edited:
Sim updated Known Bots with a new update entry:

v3.17.0 weekly bot updates

new false positives:
  • cubot r11
  • spider v7 build/lmy47i
  • spider v7 (MyCell Spider v7 from Bangladesh)

weekly bot updates:
  • adbot/1.0
  • ahrefssiteaudit
  • amazonadbot
  • amg-bot
  • ampxfbot
  • anybot
  • aranea web-crawled corpora project
  • backlinkcrawler
  • botelaire
  • chirpyhubbot
  • cookiebot
  • crawl/1.0
  • dataforseobot
  • diffeobot
  • digitalshadowsbot
  • discoverbot
  • dmasslinksafetybot
  • domcopbot
  • echocrawl
  • emeraldshield.com webbot...
Click to expand...

Read the rest of this update entry...
 
Nirjonadda said:
New bots detected [3.16.0]
  • mozilla/5.0 (linux; android 7.1; spider v7) applewebkit/537.36 (khtml, like gecko) chrome/86.0.4240.198 safari/537.36 opr/61.2.3076.56749
Click to expand...
Nirjonadda said:
New bots detected [3.16.0]

  • mozilla/5.0 (linux; android 5.1; spider v7 build/lmy47i; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/43.0.2357.121 mobile safari/537.36
  • mozilla/5.0 (compatible; ampxfbot/1.0; +http://ampxf.com/bot)
One from @mazzly addon. Identified as: Mozilla/5.0 (compatible; AMPXFBot/1.0; +http://ampxf.com/bot)
Click to expand...

Spider v7 false positives included in v3.17.0 - thanks.

AMPXFBot also included in v3.17.0
 
Nirjonadda said:
@Sim Please can be add support for https://webmeup.com/

View attachment 244785
Click to expand...

I believe Webmeup's crawler is called BLEXBot - which we already identify.

webmeup-crawler.com

SEO Software — All-In-One SEO Tools for full-cycle SEO optimization

Get an all-in-one pack of SEO software — download free edition of SEO PowerSuite. SEO tools, free SEO advice for top rankings on Google, and other search engines!
webmeup-crawler.com webmeup-crawler.com

If it's not already being identified as a bot, it could be that this is a new variant you're seeing which uses a new user agent string.

It would be useful if you could look through your web server logs to find that IP address and then identify the user agent that's using it - KnownBots works only on user agents - not on IP addresses.
 
Sim said:
If it's not already being identified as a bot, it could be that this is a new variant you're seeing which uses a new user agent string.
Click to expand...

Code: 
94.130.18.160 - - [25/Jan/2021:07:00:29 -0500] "GET / HTTP/1.1" 200 44843 "-" "Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)"
 
Potential new bots detected list:
Code: 
criteobot/0.1 (+https://www.criteo.com/criteo-crawler/)
mozilla/5.0 (compatible; amg-bot/2.1)
mozilla/5.0 (compatible; ampxfbot/1.0; +http://ampxf.com/bot)
mozilla/5.0 (compatible; testbot/0.1; +in_the_test_phase,_if_the_spider_brings_you_trouble,_please_add_our_ip_to_the_blacklist._thank_you.)
mtrobot/0.2 (metrics tools analytics crawler; https://metrics-tools.de/robot.html; crawler@metrics-tools.de)
Weirdly enough the AMPXF-bot still shows up in "potential list", but if I go to e.g. the "online stats" it is correctly placed under the Robots tab :)
 
What's your average count for robots guys?

Right now I am sitting at 119 and it feels too much. Maybe I should start blocking some.
 
sbj said:
What's your average count for robots guys?

Right now I am sitting at 119 and it feels too much. Maybe I should start blocking some.
Click to expand...

Always between 300-500. I frequently scan for malicious/dubious bots and ban them.
 
New bots detected [3.17.0]

  • mozilla/5.0 (linux; android 6.0; spider v9 build/mra58k; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/44.0.2403.119 mobile safari/537.36
 
@Sim New bots detected [3.17.0]

  • mozilla/5.0 (linux; u; android 6.0; spider v9 build/mra58k; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/44.0.2403.119 mobile safari/537.36 opr/54.0.2254.56148
  • mozilla/5.0 (linux; android 6.0; spider v9 build/mra58k; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/44.0.2403.119 mobile safari/537.36
  • mozilla/5.0 (arhpostbot/2.1; https://arhpost.ru)
 
  • Like
Reactions: Sim
Sim updated Known Bots with a new update entry:

v3.18.0 lots of new bots

new false positives:
  • cubot; j5
  • baiduboxapp
new bots:
  • 200pleasebot
  • a8bot
  • abilogicbot
  • acoonbot
  • adform robot
  • arhpostbot
  • atomseobot
  • awariorendererbot
  • badoobot
  • bl.uk_ldfc_bot
  • brobot
  • charityengine bot
  • charlotte
  • cosmos
  • coveobot
  • crawlbot/1.0.0
  • cxensebot
  • facebot
  • fandomopengraphbot
  • freshpingbot
  • fuelbot
  • geedobot
  • getlocalbot
  • google-safety
  • gpcsupbot
  • grub-client
  • gynxbot
  • healrworld crawler...
Click to expand...

Read the rest of this update entry...
 
Nirjonadda said:
New bots detected [3.17.0]

  • mozilla/5.0 (linux; android 6.0; spider v9 build/mra58k; wv) applewebkit/537.36 (khtml, like gecko) version/4.0 chrome/44.0.2403.119 mobile safari/537.36
Click to expand...

FYI - this one is actually a false positive - this is the MyCell Spider v9 mobile phone from Bangladesh.

I've added it to the false positive detection list - will be included in the next release of KnownBots.
 
You must log in or register to reply here.

Similar threads

Scandal
  • Question Question
XF 2.2 How to add new bot on known bots?
Replies
3
Views
643
Max Taxable
Max Taxable
Back
Top Bottom