Known Bots

Known Bots 6.0.3

No permission to download
Overview FAQ Updates (33) Reviews (9) History Discussion
Okay - I see the failed validation attempt in the logs.

Can you please check your validation token in the XenForo customer area - the domain on your license doesn't seem to match your forum domain. Should be an easy fix - just edit your license information and make sure your Site URL matches where your forum is installed.

1705525219520.webp

To test whether your changes work, you can force the system to send new user agents by going to Admin > Tools > Known Bots > List dectected bots and clicking on the "Send user agents" at the top of the page.
 
Sim said:
Okay - I see the failed validation attempt in the logs.

Can you please check your validation token in the XenForo customer area - the domain on your license doesn't seem to match your forum domain. Should be an easy fix - just edit your license information and make sure your Site URL matches where your forum is installed.
Click to expand...
That's odd. The domain on my license matches where our forums are. Could you please DM me what URL you are seeing? That wouldn't happen to be a URL from our separate test installation, would it?

Trying to manually send via the option at Admin > Tools > Known Bots > List dectected bots fails, stating "Sending user agents via API failed - see server logs for details", with the same sort of error in the logs as before.
 
Sim updated Known Bots with a new update entry:

6.0.3 bugfix - handle malformed UTF-8 in user agent strings

This update contains an important bugfix to handle malformed UTF-8 in user agent strings.

This update will simply ignore any user agents with malformed UTF-8, avoiding errors when trying to send updates via the API. These user agents are invalid and so there is no point undertaking any further analysis - thus they are silently discarded.

This new version also contains an additional CLI tool for importing user agents from a text file, for testing purposes.
Click to expand...

Read the rest of this update entry...
 
Bulbagarden said:
On a related note, just noticed that the sending of user agents for this bot failed yesterday.
Click to expand...

FYI - the update in KnownBots 6.0.3 should solve this issue.

Anyone who experiences a "Request error sending user agents" that doesn't give more details about the actual error - please update to 6.0.3

This error is typically caused by json_encode choking on malformed UTF-8 strings, but the Guzzle api doesn't return a meaningful error when this occurs, which made it difficult to troubleshoot.
 
@Sim
First - thanks for sharing this realy helpfull add-on with us. Thanks. :)

And now my question:
In order to reduce bad bots (and KI/AI-content-bots) I use your add-on for controling. Very helpfull.
But what I miss is the possibility to easy take a look with what IPs the found bad-bots or good-bots are coming in my forum.
In some situations this can be helpfull to block them via IP-blocking.

It would be nice to have the abitlity to klick in the list: admin.php?tools/hampel-knownbots-new at a button "IP(s)" to see a IP or list of IPs a bot is using.

greetings
 
Last edited:
otto said:
@Sim
First - thanks for sharing this realy helpfull add-on with us. Thanks. :)

And now my question:
In order to reduce bad bots (and KI/AI-content-bots) I use your add-on for controling. Very helpfull.
But what I miss is the possibility to easy take a look with what IPs the found bad-bots or good-bots are coming in my forum.
In some situations this can be helpfull to block them via IP-blocking.

It would be nice to have the abitlity to klick in the list: admin.php?tools/hampel-knownbots-new at a button "IP(s)" to see a IP or list of IPs a bot is using.

greetings
Click to expand...

You can already see this in the Members section - look at the Robots tab, which will include all bots identified by KnownBots and will also show the IP address:

1708154670141.webp
 
Yes I know, but it would better to see the IP(s) also in the last bad bots list at ACP who I can see whats going on the last 100 hours and not only the last 30 minutes. :-)
There I see a list with every bot only one time and there it would be great to have a button wich opens a list with all IPs this bot (user agent) have used last 100 hours. That can be realy helpfull to block bad bots, espacialy this with changing user agents or with user agents like members have.

I the list from your screenshot, I have to sit one ore more days in front of my PC and wrote the IPs handy on paper. I think thats not a real good solution. You dont?
 
@Sim Getting Server error log:

Code: 
Hampel\KnownBots\Exception\RequestException: Request error sending user agents: cURL error 7: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) src/addons/Hampel/KnownBots/Api/KnownBots.php:250

Generated by: Unknown account Feb 25, 2024 at 1:36 PM

Stack trace

#0 src/addons/Hampel/KnownBots/SubContainer/Api.php(109): Hampel\KnownBots\Api\KnownBots->sendUserAgents('195|Q3FUY8drkE4...', Array)
#1 src/addons/Hampel/KnownBots/Service/UserAgentSender.php(47): Hampel\KnownBots\SubContainer\Api->sendUserAgents('195|Q3FUY8drkE4...', Array)
#2 src/addons/Hampel/KnownBots/Cron/SendAgents.php(71): Hampel\KnownBots\Service\UserAgentSender->sendUserAgents()
#3 src/addons/Hampel/KnownBots/Cron/SendAgents.php(42): Hampel\KnownBots\Cron\SendAgents::sendApi(Array)
#4 src/XF/Job/Cron.php(37): Hampel\KnownBots\Cron\SendAgents::send(Object(XF\Entity\CronEntry))
#5 src/XF/Job/Manager.php(260): XF\Job\Cron->run(8)
#6 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 8)
#7 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 8)
#8 job.php(43): XF\Job\Manager->runQueue(false, 8)
#9 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(43) "/threads/95496/page-2"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
 
Nirjonadda said:
@Sim Getting Server error log:

Code: 
Hampel\KnownBots\Exception\RequestException: Request error sending user agents: cURL error 7: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) src/addons/Hampel/KnownBots/Api/KnownBots.php:250

Generated by: Unknown account Feb 25, 2024 at 1:36 PM

Stack trace

#0 src/addons/Hampel/KnownBots/SubContainer/Api.php(109): Hampel\KnownBots\Api\KnownBots->sendUserAgents('195|Q3FUY8drkE4...', Array)
#1 src/addons/Hampel/KnownBots/Service/UserAgentSender.php(47): Hampel\KnownBots\SubContainer\Api->sendUserAgents('195|Q3FUY8drkE4...', Array)
#2 src/addons/Hampel/KnownBots/Cron/SendAgents.php(71): Hampel\KnownBots\Service\UserAgentSender->sendUserAgents()
#3 src/addons/Hampel/KnownBots/Cron/SendAgents.php(42): Hampel\KnownBots\Cron\SendAgents::sendApi(Array)
#4 src/XF/Job/Cron.php(37): Hampel\KnownBots\Cron\SendAgents::send(Object(XF\Entity\CronEntry))
#5 src/XF/Job/Manager.php(260): XF\Job\Cron->run(8)
#6 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 8)
#7 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 8)
#8 job.php(43): XF\Job\Manager->runQueue(false, 8)
#9 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(43) "/threads/95496/page-2"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
Click to expand...

Request Exceptions are usually transient errors due to network or server issues.

If they only happen once, you can ignore them. If they occur more frequently (typically once per day), that's when I need to investigate.
 
First of all, thanks for this wonderful add-on! :)

For the past few days, we have been receiving the following error once a day:

Code: 
Hampel\KnownBots\Exception\ServerException: Server error sending user agents: 500 Internal Server Error src/addons/Hampel/KnownBots/Api/KnownBots.php:257

    Generated by: Unknown account 1 Mar 2024 at 15:27

Stack trace

#0 src/addons/Hampel/KnownBots/SubContainer/Api.php(109): Hampel\KnownBots\Api\KnownBots->sendUserAgents('193|Ybh8oUJgY8x...', Array)
#1 src/addons/Hampel/KnownBots/Service/UserAgentSender.php(47): Hampel\KnownBots\SubContainer\Api->sendUserAgents('193|Ybh8oUJgY8x...', Array)
#2 src/addons/Hampel/KnownBots/Cron/SendAgents.php(71): Hampel\KnownBots\Service\UserAgentSender->sendUserAgents()
#3 src/addons/Hampel/KnownBots/Cron/SendAgents.php(42): Hampel\KnownBots\Cron\SendAgents::sendApi(Array)
#4 src/XF/Job/Cron.php(37): Hampel\KnownBots\Cron\SendAgents::send(Object(XF\Entity\CronEntry))
#5 src/XF/Job/Manager.php(260): XF\Job\Cron->run(8)
#6 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 8)
#7 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 8)
#8 job.php(43): XF\Job\Manager->runQueue(false, 8)
#9 {main}
 
Are you on the current version? An issue was fixed in the past version where it could crash daily when sending the bot report to the host.
 
I also have one of those logged, on Mar 6, 2024 at 10:42 GMT+2 - if that helps you dig through your logs to find the event in question :)
 
I'm investigating whether this is a problem at my end or a problem at the sending end - will report back.
 
I've deployed an update to my server to try and capture the bad data so I can test it and identify what data is making the system fail.

Definitely looks like a problem at my end rather than the sending end.
 
thomas1 said:
First of all, thanks for this wonderful add-on! :)

For the past few days, we have been receiving the following error once a day:

Code: 
Hampel\KnownBots\Exception\ServerException: Server error sending user agents: 500 Internal Server Error src/addons/Hampel/KnownBots/Api/KnownBots.php:257

    Generated by: Unknown account 1 Mar 2024 at 15:27

Stack trace

#0 src/addons/Hampel/KnownBots/SubContainer/Api.php(109): Hampel\KnownBots\Api\KnownBots->sendUserAgents('193|Ybh8oUJgY8x...', Array)
#1 src/addons/Hampel/KnownBots/Service/UserAgentSender.php(47): Hampel\KnownBots\SubContainer\Api->sendUserAgents('193|Ybh8oUJgY8x...', Array)
#2 src/addons/Hampel/KnownBots/Cron/SendAgents.php(71): Hampel\KnownBots\Service\UserAgentSender->sendUserAgents()
#3 src/addons/Hampel/KnownBots/Cron/SendAgents.php(42): Hampel\KnownBots\Cron\SendAgents::sendApi(Array)
#4 src/XF/Job/Cron.php(37): Hampel\KnownBots\Cron\SendAgents::send(Object(XF\Entity\CronEntry))
#5 src/XF/Job/Manager.php(260): XF\Job\Cron->run(8)
#6 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 8)
#7 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 8)
#8 job.php(43): XF\Job\Manager->runQueue(false, 8)
#9 {main}
Click to expand...
DragonByte Tech said:
I also have one of those logged, on Mar 6, 2024 at 10:42 GMT+2 - if that helps you dig through your logs to find the event in question :)
Click to expand...

I've deployed an update to my server which should stop these errors.

*grumble* stupid UTF-8 strings *grumble*
 
You must log in or register to reply here.

Similar threads

Scandal
  • Question Question
XF 2.2 How to add new bot on known bots?
Replies
3
Views
627
Max Taxable
Max Taxable
Top Bottom