![[TAC] Bot Arrestor](/community/data/resource_icons/3/3305.jpg?1577369073){kind=icon}
[TAC] Bot Arrestor [Paid] 2.0.12
- Thread starter tenants
- Start date
tenants
Well-known member
I'm currently looking at ajax calls causing extra page counts (for instance, hovering over the link calling the thread preview).
It's probably not a good idea to count these (since the user has no idea they are really 'visiting' an extra page).
For threads, I now only increase the count when actually visiting the page (actionIndex), I might have to do this for other areas to
I've added a new version:
DeDos_v1_0_9b.zip
This should also avoid counting things like the unread redirect for threads, I'm looking at other areas people might call a lot of ajax requests or redirects.
- The aim of this plugins isn't so much to stop human DOS attacks, as it is to not interfere with humans/spiders (yet still catch resource selfish scrapers & spam bots)
It's probably not a good idea to count these (since the user has no idea they are really 'visiting' an extra page).
For threads, I now only increase the count when actually visiting the page (actionIndex), I might have to do this for other areas to
I've added a new version:
DeDos_v1_0_9b.zip
This should also avoid counting things like the unread redirect for threads, I'm looking at other areas people might call a lot of ajax requests or redirects.
- The aim of this plugins isn't so much to stop human DOS attacks, as it is to not interfere with humans/spiders (yet still catch resource selfish scrapers & spam bots)
tenants
Well-known member
tenants updated DeDos - Anti DOS for spam bot/scrapers with a new update entry:
avoid logged in
Read the rest of this update entry...
Instead of upgrading, I would un-install and reinstall this version (to use the new settings)
avoid logged in
Read the rest of this update entry...
Instead of upgrading, I would un-install and reinstall this version (to use the new settings)
tenants
Well-known member
StopBotResources is no longer needed when using FBHP and DeDos together (it has been removed).
StopBotResources relied on an API for every single visitors.. That means the more forums that used the plugin, the more the API became saturated with requests (which is why I only allowed small forums to use that resource and asked admins to remove the plugin once they no longer met certain conditions)...
Using FoolBotHoneyPot & DeDos together we have a better approach
FoolBotHoneyPot detects spam bots registering (currently 100% of spam bots)
DeDos detects spam bots hitting your site heavily
Once detected, they both use the local cache... so we have a very low query approach (and low bandwidth) of locking these spam bot / scrapers out
- There is no need for an API, so yes, using both together they can help reduce bot resources on servers with limited resources / high amount of spam bots, and they can be used on any size forum
StopBotResources relied on an API for every single visitors.. That means the more forums that used the plugin, the more the API became saturated with requests (which is why I only allowed small forums to use that resource and asked admins to remove the plugin once they no longer met certain conditions)...
Using FoolBotHoneyPot & DeDos together we have a better approach
FoolBotHoneyPot detects spam bots registering (currently 100% of spam bots)
DeDos detects spam bots hitting your site heavily
Once detected, they both use the local cache... so we have a very low query approach (and low bandwidth) of locking these spam bot / scrapers out
- There is no need for an API, so yes, using both together they can help reduce bot resources on servers with limited resources / high amount of spam bots, and they can be used on any size forum
Alpha1
Well-known member
what is the difference between this and bad behavior?
does this addon recognize suspect browser fingerprints like for example ie6?
does this work with chat software which polls the server a few times every second?
I run litespeed Web server which has settings for blocking ips that connect too often. I had to increase that limit to allow chat users not to be banned by litespeed.
does this addon recognize suspect browser fingerprints like for example ie6?
does this work with chat software which polls the server a few times every second?
I run litespeed Web server which has settings for blocking ips that connect too often. I had to increase that limit to allow chat users not to be banned by litespeed.
tenants
Well-known member
For a long time, spam bots have been faking user_agent. If you use FoolBotHoneyPot, you will see that every bot, although they have no JS, attempt to register within seconds, get detected on various APIs and alter 15-21 different types of hidden fields, every single one fakes the user_agent to look like a modern browser.
In DeDos, we actually use this to our advantage, we can ignore all spiders by saying that we only block users that look like browsers (and DOS), DeDos also looks for known spiders using core functionality (and avoid detecting them). So in away, the spam bots 'disguise' is it's own downfall... if the spam bot faked their selves as googlebot, then we would have no way of distinguishing them (thankfully spam bots want to look like humans when the attempt to sign up).
- Avoid false positives humans/spiders is the direction this is going (much like FBHP, FBHP current blocks 100% of spam bots with no false positives and users never notice it)
If we start adding options anywhere to block user_agent browsers types, this will only ever affect your humans with false positives (this is bad design), user_agent can be (and is always) faked. This is nothing like bad behaviour, I'll do everything I can to avoid false positives, I will not add options to hang your self with false positives, I may add options that will avoid them ...
DeDos (and FoolBotHoneyPot) are targeted at spam bots & scrappers without real human ever noticing an impact.
This is still an early version of DeDos, there are a couple more options I would like to add
Since this is focusing on Bots (and not humans), next I will add an options so that only JavaScript disabled users are added to the cache (humans will still get the lower query/lower kb warning, but wont ever get added to the cache)
For chat software:
... It depends how the chat software works, I suspect DeDos will ignore the chat software since DeDos is primarily focused at detecting DOS for core functions (having said that, if the plugin extends the core functional areas, it might not be ignored...) is there an example of the chat software (If it's free, I can take a look and tell you quite quickly)
In DeDos, we actually use this to our advantage, we can ignore all spiders by saying that we only block users that look like browsers (and DOS), DeDos also looks for known spiders using core functionality (and avoid detecting them). So in away, the spam bots 'disguise' is it's own downfall... if the spam bot faked their selves as googlebot, then we would have no way of distinguishing them (thankfully spam bots want to look like humans when the attempt to sign up).
- Avoid false positives humans/spiders is the direction this is going (much like FBHP, FBHP current blocks 100% of spam bots with no false positives and users never notice it)
If we start adding options anywhere to block user_agent browsers types, this will only ever affect your humans with false positives (this is bad design), user_agent can be (and is always) faked. This is nothing like bad behaviour, I'll do everything I can to avoid false positives, I will not add options to hang your self with false positives, I may add options that will avoid them ...
DeDos (and FoolBotHoneyPot) are targeted at spam bots & scrappers without real human ever noticing an impact.
This is still an early version of DeDos, there are a couple more options I would like to add
Since this is focusing on Bots (and not humans), next I will add an options so that only JavaScript disabled users are added to the cache (humans will still get the lower query/lower kb warning, but wont ever get added to the cache)
For chat software:
... It depends how the chat software works, I suspect DeDos will ignore the chat software since DeDos is primarily focused at detecting DOS for core functions (having said that, if the plugin extends the core functional areas, it might not be ignored...) is there an example of the chat software (If it's free, I can take a look and tell you quite quickly)
Last edited:
kontrabass
Well-known member
Got a couple errors today.
Code:
Error Info
ErrorException: Fatal Error: Call to undefined method XFCP_Tac_DeDos_ControllerPublic_Misc::actionIndex() - library/Tac/DeDos/ControllerPublic/Misc.php:9
Generated By: Unknown Account, 41 minutes ago
Stack Trace
#0 [internal function]: XenForo_Application::handleFatalError()
#1 {main}
Request State
array(3) {
["url"] => string(29) "http://www.talkbass.com/misc/"
["_GET"] => array(0) {
}
["_POST"] => array(0) {
}
}kontrabass
Well-known member
@tenants Do you believe this addon could cause load issues on large forums? Just wondering - we've had some "clogging" in the pipes since installing this, and not sure if it's completely unrelated. These "clogs" happen once every 15-30 mins or so: pages don't load for about 8-10 seconds, then everything starts loading fine (webserver load jumps to 10 when it becomes "unclogged" and then settles back down). Server techs say the database server isn't having any issues. Hmm.
tenants
Well-known member
There are no extra database queries, it simply adds to the session, and adds to the cache if found as a spam bot (the data is tiny, bytes).
So for large forums, it's not going to add thousands of IPs to the global cache. Instead, each user session will have a small amount of data (from the last 10 seconds, and no more than x attempts).
Each user, in there sessions data, will have something like this
dd_recent_visits";s:125:"a:1:{i:0;a:2:{s:1:"t";i:1402556250;s:1:"l";s:72:"http://localhost/xenforo/index.php?threads/blabla.3378/";}}"
[This is a small amount of data, and only related to that user for the last 10 seconds]
If and only if this data is found as a DOS attempt, then the data is added to the global cache (once again, no extra queries)
This data is stored in the global cache in it's smallest form [ip]=timecached
"dd_known_dos";a:1:{s:9:"127.0.0.1";i:1402493810;}
[This does not build up, since the data is removed once older than the cache time range]
So, not only is it 0 query overhead, the data stored is very small (per user session for recent visits, global cache for known DOS)
The DOS check is called once per page, so it simply grabs dd_known_dos and checks to see if the IP is present (its a very simple calculation to check if a key exists in a small size array, so this is very low PHP overhead).
So no, this has been aimed at being
So for large forums, it's not going to add thousands of IPs to the global cache. Instead, each user session will have a small amount of data (from the last 10 seconds, and no more than x attempts).
Each user, in there sessions data, will have something like this
dd_recent_visits";s:125:"a:1:{i:0;a:2:{s:1:"t";i:1402556250;s:1:"l";s:72:"http://localhost/xenforo/index.php?threads/blabla.3378/";}}"
[This is a small amount of data, and only related to that user for the last 10 seconds]
If and only if this data is found as a DOS attempt, then the data is added to the global cache (once again, no extra queries)
This data is stored in the global cache in it's smallest form [ip]=timecached
"dd_known_dos";a:1:{s:9:"127.0.0.1";i:1402493810;}
[This does not build up, since the data is removed once older than the cache time range]
So, not only is it 0 query overhead, the data stored is very small (per user session for recent visits, global cache for known DOS)
The DOS check is called once per page, so it simply grabs dd_known_dos and checks to see if the IP is present (its a very simple calculation to check if a key exists in a small size array, so this is very low PHP overhead).
So no, this has been aimed at being
- low query overhead (0)
- low data size (bytes)
- low PHP calculation (check if key exists in a small array)
Last edited:
BamBam
Active member
Bought & installed!
One Question: I have tested it for myself, but the only Error Message i get is this: http://awesomescreenshot.com/0192ywc799 - there was no "friendly" Message with a Warning. I dont have changed the Settings, is all Default. How can i enable a Warning first?
// Ah, nevermind. I saw that Avoid Logged In Option is there, so Logged in Users are avoides by this Addon. Nice!
One Question: I have tested it for myself, but the only Error Message i get is this: http://awesomescreenshot.com/0192ywc799 - there was no "friendly" Message with a Warning. I dont have changed the Settings, is all Default. How can i enable a Warning first?
// Ah, nevermind. I saw that Avoid Logged In Option is there, so Logged in Users are avoides by this Addon. Nice!
Last edited:
tenants
Well-known member
Yes, I'll avoid JavaScript enabled users too soon (so even real human users that are logged out should also never get added to the cache, but may still see the human friendly message if triggered)
To see the friendly message first, make sure it is triggered with less requests than the malicious settings (I think I may have set the number of requests by default to the same...that's not a good idea)
For instance, I've now changed the defaults to:
Friendly
7 Requests in 8 Seconds
Malicious (added to the cache)
9 Requests in 6 Seconds
This means, refreshing the page 7 times you'll see the friendly warning message (on the 9th, you'll get added to the cache if you have done this in under 6 seconds)
It also depends on where you refresh, what page was this?
For most areas I catch the actionIndex with the friendly message (this is usually related to the page, for instance the thread page, post page, conversation page), but I don't prompt the friendly message for other areas related to hover-over/ajax, and now avoid checking things such as preview posts (these no longer add to the recentVisits).
To see the friendly message first, make sure it is triggered with less requests than the malicious settings (I think I may have set the number of requests by default to the same...that's not a good idea)
For instance, I've now changed the defaults to:
Friendly
7 Requests in 8 Seconds
Malicious (added to the cache)
9 Requests in 6 Seconds
This means, refreshing the page 7 times you'll see the friendly warning message (on the 9th, you'll get added to the cache if you have done this in under 6 seconds)
It also depends on where you refresh, what page was this?
For most areas I catch the actionIndex with the friendly message (this is usually related to the page, for instance the thread page, post page, conversation page), but I don't prompt the friendly message for other areas related to hover-over/ajax, and now avoid checking things such as preview posts (these no longer add to the recentVisits).
Last edited:
tenants
Well-known member
tenants updated DeDos - Anti DOS for spam bot/scrapers with a new update entry:
bug fix, clean up cache
Read the rest of this update entry...
bug fix, clean up cache
Read the rest of this update entry...
Floyd R Turbo
Well-known member
Dude, the way you explain how your add-ons work makes me want to fork over dough without thought. Already have FBHP on one site, now I want this too and another copy of both for my other site. Dang!!
tenants
Well-known member
tenants updated DeDos - Anti DOS for spam bot/scrapers with a new update entry:
Many Enhancements - detect cookless bots
Read the rest of this update entry...
Many Enhancements - detect cookless bots
Read the rest of this update entry...
Last edited:
tenants
Well-known member
Hmm, I'm not able to reproduce it yet on 3 forums:
http://xenzine.com/search/member?user_id=1
are there any error logs, can you send me your url
http://xenzine.com/search/member?user_id=1
are there any error logs, can you send me your url
