fun so far. this addon seems to basically run the site in under attack mode. the only difference is that you have control over the the captcha like you can restrict it on threads. and logged in users do not face any captcha!
XF Bot Guard 1.4.1
- Thread starter zeeb0t
- Start date
I’ve also found that some bots do register accounts, but in my experience it’s only a small number.
For what it’s worth, I run it with the following settings and find that the majority of genuine users are not impacted:
- Guests and registered users enabled
- IP exclusions for my own uptime monitoring, so it always has full access
- Allow known verified crawler headers where they can be determined
Everything else is left on the default settings.
Great to hear you’re enjoying it and finding it so effective.
Truth be told, I’ve spent a lot of time over the years understanding how sophisticated bots work, and building them for my employers. These days I’m putting that knowledge to better use - fighting the good fight and helping protect our forums.
zeeb0t updated XF Bot Guard with a new update entry:
Release: v1.0.5
Read the rest of this update entry...
Release: v1.0.5
Read the rest of this update entry...
I’ve been reviewing the current state of the add-on after the 1.0.5 hardening work, and I wanted to share where I think the next release is heading.
The next release, 1.0.6, is planned to be the first stable release of XF Bot Guard.
That does not mean the roadmap is finished. Far from it. There is still a lot I want to build into this add-on over time, including stronger behavioural analysis, better operator tooling, more advanced route and session-shape intelligence, and further anti-scraping features.
But I think the add-on has now reached the point where the right next step is not to keep piling on new detection ideas immediately. The priority for 1.0.6 is to harden the existing foundation so it is ready for serious forums.
The focus for 1.0.6 will be stability, performance, safer defaults, cleaner logging, improved health checks, better operational visibility, and making sure the 1.0.5 hardening work behaves reliably in real XenForo deployments.
In other words, 1.0.5 was the major hardening milestone. 1.0.6 is intended to be the release that takes XF Bot Guard from hardened beta to stable production use.
XF Bot Guard still should not be treated as a replacement for Cloudflare, a WAF, or server-level protection. It runs inside XenForo and is designed to act as a XenForo-aware application-layer defence against scraping, suspicious guest browsing, no-JS fetchers, sloppy bots, and opportunistic crawlers.
The longer-term roadmap remains very active, but I want the first stable release to be boring in the right ways: reliable, predictable, easier to monitor, safer to deploy, and suitable for forums that need a serious native anti-scraping layer without depending on an external bot-detection SaaS.
That is the goal for 1.0.6.
The next release, 1.0.6, is planned to be the first stable release of XF Bot Guard.
That does not mean the roadmap is finished. Far from it. There is still a lot I want to build into this add-on over time, including stronger behavioural analysis, better operator tooling, more advanced route and session-shape intelligence, and further anti-scraping features.
But I think the add-on has now reached the point where the right next step is not to keep piling on new detection ideas immediately. The priority for 1.0.6 is to harden the existing foundation so it is ready for serious forums.
The focus for 1.0.6 will be stability, performance, safer defaults, cleaner logging, improved health checks, better operational visibility, and making sure the 1.0.5 hardening work behaves reliably in real XenForo deployments.
In other words, 1.0.5 was the major hardening milestone. 1.0.6 is intended to be the release that takes XF Bot Guard from hardened beta to stable production use.
XF Bot Guard still should not be treated as a replacement for Cloudflare, a WAF, or server-level protection. It runs inside XenForo and is designed to act as a XenForo-aware application-layer defence against scraping, suspicious guest browsing, no-JS fetchers, sloppy bots, and opportunistic crawlers.
The longer-term roadmap remains very active, but I want the first stable release to be boring in the right ways: reliable, predictable, easier to monitor, safer to deploy, and suitable for forums that need a serious native anti-scraping layer without depending on an external bot-detection SaaS.
That is the goal for 1.0.6.
What I'd like to see in conjunction with this is the thing playing nicely along with the IP Threat Manager by @Osman . The two could potentially be a very good team - or in opposite hinder each other.
Is there any issue you have identified with the two working together already? I suspect at minimum we could coordinate the order of template matters - e.g., any IP blocked is not worth considering by XF Bot Guard. @Osman if you are keen to chat, let me know.
No, I haven't tried it. On my test/staging environment it would not make much sense, as apart from a potential spotanous explosion I would not recognize issues due to lack of load and crawlers. On production it is to risky to try on the other hand.
Fair, well the next version will have an optimal approach from a load management perspective. Plus for instance, a health check page within ACP which will give you a full run-down if there are any detected issues with the add-on performing as it should. I will provide further details in the next day or two once it is ready. P.s., this stable release will also introduce performance time records. It won't feature a dashboard report (for a future release) but it will be queryable and viewable at a micro level for now.
zeeb0t updated XF Bot Guard with a new update entry:
Stable Release: v1.0.6
Read the rest of this update entry...
Stable Release: v1.0.6
Read the rest of this update entry...
XF Bot Guard 1.0.6 is now the first stable release! This version is already running in production on my own forum and has been through real traffic, real bot pressure, real CAPTCHA flows, and real operational use. The stable label is not just theoretical - this release is based on the foundation that has already been doing the job in production.
That said, every XenForo forum is different. Different themes, CAPTCHA providers, Cloudflare/proxy setups, traffic patterns, add-ons, and server environments can expose things that do not appear on one production site.
So for the next week or two, I am going to let 1.0.6 settle before moving straight into the next feature cycle.
The focus during this period is production support and real-world feedback from other forums using the stable release.
If you install or upgrade to 1.0.6, please keep an eye on the new health/status page, the event log, CAPTCHA behaviour, normal guest browsing, and any unexpected challenge patterns.
Useful feedback includes:
- XenForo version
- PHP version
- CAPTCHA provider
- Whether Cloudflare/proxy/CDN is in use
- Relevant Bot Guard settings
- Relevant event log details, risk score, and reason codes
- Whether the visitor was a guest, member, or staff
- Whether the issue happened before or after CAPTCHA
The goal is not to rush another feature release immediately. The goal is to make sure the stable foundation behaves cleanly across more real forums, with different configurations and real-world traffic.
After this settling period, the next work is likely to focus on better operator visibility and more intelligent browsing-shape detection. That may include improved event-log filtering, clearer challenge/reason summaries, top reason and challenge-pressure views, threshold review tools, and conservative detection of scraping-like reading patterns.
For now, 1.0.6 is the recommended stable foundation. Please run it, watch what it does, and report anything that looks wrong, confusing, too aggressive, too quiet, or worth improving.
Where would I find these? As before, I still can't find any log or status location for the add on.
Also, on upgrade from 1.0.2 to 1.0.6 on my test environment I got the following error:
Thanks for reporting this.
That error means the xf_zee_botguard_online_challenge table was not present when the add-on’s cleanup process tried to run. That table should exist in 1.0.6, so it looks like the upgrade from 1.0.2 to 1.0.6 did not fully apply the later database changes in your test environment.
For now, please do a clean reinstall of 1.0.6:
- Uninstall XF Bot Guard
- Install the 1.0.6 release fresh
Please note that uninstalling may remove the add-on’s stored data/settings depending on your environment, so if this is not just a test install, make a note of your settings first.
The areas you are looking for are:
- Options: admin.php?add-ons/Zee-BotGuard/options
- Event log: admin.php?bot-guard-log/ - which also links through to:
- Health/status check: admin.php?bot-guard-log/health
I’ll also review the non-sequential upgrade path so that larger jumps such as 1.0.2 → 1.0.6 are handled more safely in a future maintenance release.
Thanks, this worked.
Ah, indeed:
This is however not really intuitive. Is there a reason why you choose this? The Logs should rather be in "ACP -> Logs" in my opinion.
Yeah, that’s a fair criticism.
I kept it self-contained because one of my own pet peeves is add-ons placing links all over the ACP. My preference was to have the Bot Guard entry act as the central place for options, logs, and health/status.
But I agree that ACP → Logs is probably the more intuitive place for an event log, so I’ll look at adding it there too.
What I am currently missing is how this add on deals with search engine crawlers. IP Thread monitor has the option to let certain bots pass w/o interruption, configurable and verified via various ways, i.e. via their published IP ranges and rDNS.
I consider this really helpful (if not mandatory) and this is i.e. an area where the two add ons could stack upon each other to avoid double checking.
I consider this really helpful (if not mandatory) and this is i.e. an area where the two add ons could stack upon each other to avoid double checking.
Yeah, I agree this is within the wheelhouse of XF Bot Guard.
The main reason it isn’t there yet is that I don’t want to ship a static or half-maintained trusted-bot list. Search engine IP ranges and verification methods can change over time, so this is something I’d want to thoroughly and carefully automate.
It is roadmapped, but I want it implemented properly - using reliable updates, rDNS verification where appropriate, caching, and clear admin controls - rather than adding something that may quietly become stale as I build out further features.
For now, XF Bot Guard leans on Cloudflare’s verified bot signal where Cloudflare is in front of the site. You can find more information on setting that up in the FAQ.
Yes. That's what I meant with "IP Threat monitor and Bot Guard could profit from each other". Why reinvent the wheel if it is already there.
I don't use Cloudflare and do not really plan to use it either for the moment. That is the main reason why I am interested in the add on.
On a feature level:
What could be helpful would be a statistic:
• how many visitors overall
• how many have passed w/o being challenged
• how many have been challenged
• how many mastered the challenge successfully
• how many failed the challenge
Also a possible point for the FAQ: What happens if a visitor fails to complete the challenge successfully?
• Will he be sitting in front of the challenge literally "for ever"?
• Will he get a 403?
etc.
I gave it a shot and installed the add on on production (along with IP threat Monitor) while monitoring closely. Nothing exploded. Quite a bunch of Visitors got challenged immediately (typically with risk scores of 75 and 85) and others only observed. So far so good, let's see how it goes.
Yep, I am hoping to hear from the IP Threat Detection dev soon. It would definitely be good if the two add-ons could work together in that way. I just do not want to open up too many lanes too quickly by building it myself right now, especially while Bot Guard is still early and the core behaviour needs to stay predictable.
Good suggestions on the stats. That kind of dashboard/reporting view is already planned for the next release, and those are exactly some of the sorts of figures I would want surfaced.
Re: Cloudflare, fair enough. Bot Guard does not require Cloudflare (with exception to optional known good bots), and I completely understand not wanting to use it. Cloudflare still has value for things like edge caching, blocking at the edge, DDoS protection, etc., but whether that is worth using depends on the site.
For the FAQ question: if a visitor does not complete the challenge, they remain at the challenge gate. They do not get a 403 just because they failed to complete it. They would need to successfully complete the challenge before reaching the protected route.
The only real “reset” is that the fingerprint/session state can eventually expire, after which they may be assessed again on a future visit. If Bot Guard still detects the same behaviour, they will simply be challenged again.
That's great to hear! My own experience has been that it detects and challenges (effectively blocking) the vast majority of non-human traffic, even at the current feature level. Soon enough, even the most sophisticated bots will be challenged
What's worth mentioning is the order in which the add ons work: Based on my testing it seems, that IP Threat Monitor hits first and only if a visitor has managed to pass it Bot Guard comes into play. Fine for me. Even more curious on the statistics now as this way I could find out what amount of what IP Threat Monitor let's trough get's then caught by Bot Guard.
Even more curious on the statistics now as this way I could find out what amount of what IP Threat Monitor let's trough get's then caught by Bot Guard.
