Known Bots 6.0.3

[pecadm]

The Claudebot seems to find our board interesting. It is currently there with around 500+ instances at the same time.
However, it is not identified as a bot, but as a guest.
I can't find it in the list of known bots. But I can't find a way to add it manually either.
"Fetch new bots" has no updates, but in the list of known bots it says "Last updated: Wed, 14 Feb 2024 04:59:15 +0100".
Installed version is 6.0.3, no error messages or similar.

Any idea what I can do to have the bot identified as a bot?

 

[wolfgangm]

Thanks for the information! I also have 500+ guests at the moment and meanwhile for some days, all at the main page.

No idea what to do?

Wolfgang

 

[alexm]

The Claudebot seems to find our board interesting. It is currently there with around 500+ instances at the same time.
However, it is not identified as a bot, but as a guest.
I can't find it in the list of known bots. But I can't find a way to add it manually either.
"Fetch new bots" has no updates, but in the list of known bots it says "Last updated: Wed, 14 Feb 2024 04:59:15 +0100".
Installed version is 6.0.3, no error messages or similar.

Any idea what I can do?

Us too. I started adding IPs to the firewall and gave up in the end, they kept on coming.

 

[pecadm]

Perhaps my question was misleading.
My point is to identify the bot as a bot.
I have adapted my question in the original post.

For tips on how bots can be blocked, there are already tips here: https://xenforo.com/community/threads/known-bots.148723/post-1662244

 

[Sim]

Any idea what I can do to have the bot identified as a bot?

Posting in this thread is the best way - I can go through my list of pending user agents and see if it's a bot I haven't yet processed.

In this case it was indeed in my database in the pending list - I have around 15,000 user agent strings to process, most of them valid browsers - when I get time I go through and add a few more bots to the database.

So next time your system pulls the update (or you can force an update now), it should include a definition for ClaudeBot.

 

[pecadm]

I just successfully updated the list manually, but the claudebot isn't in there yet.

 

[wolfgangm]

Today in the evening the bot stopped on my site, now the usual 20 guests (instead of 500+ the days before).
Seems they stopped it, maybe our company firewall? I didn't do anything.

Wolfgang

 

[Sim]

I just successfully updated the list manually, but the claudebot isn't in there yet.

View attachment 300457

View attachment 300458

The update probably hadn't come through yet - there is some caching involved at certain points.

I forced an update on one of my sites and it's there now:

 

[philmckrackon]

I got pounded by Bytespider and Baiduspider yesterday with over 3,000 of them at once. This is what I put in my .htacess file. There may be better ways of blocking them but it worked.

BrowserMatchNoCase "Bytedance" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
Order Deny,Allow
Deny from env=bad_bot

 

[Opus X]

I got pounded by Bytespider and Baiduspider yesterday with over 3,000 of them at once. This is what I put in my .htacess file. There may be better ways of blocking them but it worked.

if you use cloudflare block them at that level and they never even get to your site.

 

[philmckrackon]

if you use cloudflare block them at that level and they never even get to your site.

I do not use cloudflare, so it's just .htaccess for me.

 

[avalanch]

I have 2 new ones coming to visit me... Does anyone know anything about them as in what their parent sites are about?

*ZoominfoBot
*DataForSEO Link Bot

 

[avalanch]

if you use cloudflare block them at that level and they never even get to your site.

Yes cloudflare is fantastic for this, just throw in their user agent & set it to block... you'll never see that bot again plus it will stop the bot from using any of your hosting resources as cloudflare tackles it before it can touch your host.

 

[Opus X]

Yes cloudflare is fantastic for this, just throw in their user agent & set it to block... you'll never see that bot again plus it will stop the bot from using any of your hosting resources as cloudflare tackles it before it can touch your host.

i do not maintain my site cloudflare configuration but the person who does keeps up with that closely.
i do not get many robots hitting the site other than the good ones.

 

[avalanch]

i do not maintain my site cloudflare configuration but the person who does keeps up with that closely.
i do not get many robots hitting the site other than the good ones.

That's not a good idea to start with... letting someone else handle your websites cloudflare. For instance, they could ip ban you anytime they like with cloudflare.. It is your website right?

 

[Opus X]

That's not a good idea to start with... letting someone else handle your websites cloudflare. For instance, they could ip ban you anytime they like with cloudflare.. It is your website right?

he could do that anyway as my site runs on his server.
after knowing him over a decade i doubt that he would do that since he also sold the site to me for $25.