Last edited:
XF Bot Guard 1.4.1
- Thread starter zeeb0t
- Start date
Do you mean a link to the Dashboard? Or the entire Dashboard?
Entire dashboard, stats one.
Thank you @zeeb0t - Just as a heads up, I'm on a dedicated server, not shared
In your case, you may be able to change your config to allow PHP to increase its running time limit. But, there's also a new update coming our shortly that resolves it for all.
XF Bot Guard 1.3.8 is currently being production tested and, assuming no surprises appear, will be released soon.
At a high level, 1.3.8 includes:
In plain English: this release should make Bot Guard quieter, more efficient, more transparent, and easier to run across a wider range of hosting environments.
Full release notes will follow when 1.3.8 is ready.
At a high level, 1.3.8 includes:
- Smarter background work: long-running cron jobs are now more budget-aware, resumable, and easier to diagnose through Health Check visibility.
- Privacy-assured protection mode: a new deployment option for sites that want a more privacy-conscious/GDPR-compatible Bot Guard setup.
- Better Light-DB dashboard behaviour: unsupported panels are hidden where they cannot be accurately populated, and Light-DB-friendly reporting has been improved.
- Known crawler lookup optimisation: crawler IP range checks are now more compact and efficient where XenForo application cache is available.
- Updated Web Bot Auth beta fallback: alignment work for the newer replacement IETF draft direction.
- Link Chronology optimisation: protected link target state is now de-duplicated more efficiently.
- General performance and cleanup work: many CPU, memory, database, cache, and internal refactoring improvements across the add-on.
In plain English: this release should make Bot Guard quieter, more efficient, more transparent, and easier to run across a wider range of hosting environments.
Full release notes will follow when 1.3.8 is ready.
zeeb0t updated XF Bot Guard with a new update entry:
Minor Release: v1.3.8
Read the rest of this update entry...
Minor Release: v1.3.8
Read the rest of this update entry...
A note on GDPR / the "privacy-assured" mode now available in XF Bot Guard:
Bot Guard now provides a GDPR-compatible operating mode called Privacy-assured protection mode. When enabled, it reduces passive browser fingerprinting and persistent browser storage on ordinary page views while still allowing enhanced browser validation when protected traffic needs proof. Pair it with Light-DB runtime mode when database minimisation is also required.
This supports a privacy-conscious security posture, but it is not a legal compliance guarantee by itself, so your site’s privacy, cookie and storage notices should still reflect your actual configuration.
---
With the new feature option, Bot Guard is intended to support security processing under GDPR Article 6(1)(f), where a forum has a legitimate interest in protecting its service from abuse, automated scraping, credential attacks, spam, denial-of-service pressure and other malicious or unlawful activity. Recital 49 recognises strictly necessary and proportionate processing for network and information security as a legitimate interest, and Article 32 requires appropriate technical and organisational measures to protect the confidentiality, integrity, availability and resilience of systems. Sites should still disclose this processing in their privacy/cookie notices under Article 13, apply the Article 5 principles of transparency, minimisation and storage limitation, and consider any ePrivacy rules that apply to cookies, localStorage or browser fingerprinting.
Sources: GDPR Articles 5, 6, 13, 32 and Recital 49 on EUR-Lex; EDPB Guidelines 2/2023 on ePrivacy Article 5(3).
Bot Guard now provides a GDPR-compatible operating mode called Privacy-assured protection mode. When enabled, it reduces passive browser fingerprinting and persistent browser storage on ordinary page views while still allowing enhanced browser validation when protected traffic needs proof. Pair it with Light-DB runtime mode when database minimisation is also required.
This supports a privacy-conscious security posture, but it is not a legal compliance guarantee by itself, so your site’s privacy, cookie and storage notices should still reflect your actual configuration.
---
With the new feature option, Bot Guard is intended to support security processing under GDPR Article 6(1)(f), where a forum has a legitimate interest in protecting its service from abuse, automated scraping, credential attacks, spam, denial-of-service pressure and other malicious or unlawful activity. Recital 49 recognises strictly necessary and proportionate processing for network and information security as a legitimate interest, and Article 32 requires appropriate technical and organisational measures to protect the confidentiality, integrity, availability and resilience of systems. Sites should still disclose this processing in their privacy/cookie notices under Article 13, apply the Article 5 principles of transparency, minimisation and storage limitation, and consider any ePrivacy rules that apply to cookies, localStorage or browser fingerprinting.
Sources: GDPR Articles 5, 6, 13, 32 and Recital 49 on EUR-Lex; EDPB Guidelines 2/2023 on ePrivacy Article 5(3).
Last edited:
You could try - but bad bots generally don't give a hoot about your robots exclusions.
The better option would be to remove any GA tracking on the page (if possible) or exclude them inside GA settings (e.g. custom report filters)
This is "expected" behaviour - in that, bots are going to end up there, a LOT. So much so that exceed even home page hits. This is a good sign if you have lots of JS-enabled bots, and have Bot Guard installed.
That said, I see how I can actually create a default feature in Bot Guard so that it excludes embedding the GA script inside the bot-guard/ paths. I am going to include this in the next release, so that the noise gets removed from your GA reports.
Last edited:
Just a note on my system running the recent Bot Guard updates. Cloudflare numbers show the site is still under intermittent attacks which would previously show 10,000+ visitors, but the site stats show what appears to be the accurate actual visitors, less than 1000. Site is performing well, the database load has decreased dramatically when running light DB mode. On early Bot Guard installs database CPU increased to average around 25%, then updates brought it down to 15%. Running the latest with light DB now I see under 10%. Thanks again for the great work @zeeb0t!
seem to have an error appeared @zeeb0t. Anything to worry about?? Just started (in the last couple of hours) running 1.3.8
#1 src/addons/Zee/BotGuard/Service/KnownCrawlerRefresher.php(39): Zee\BotGuard\Service\KnownCrawlerRefresher->seedSourcesIfNeeded()
#2 src/addons/Zee/BotGuard/Cron/KnownCrawlerRefresh.php(29): Zee\BotGuard\Service\KnownCrawlerRefresher->refreshAll(false, Object(Zee\BotGuard\Service\CronRunBudget))
#3 src/addons/Zee/BotGuard/Service/HealthCriticalMaintenance.php(240): Zee\BotGuard\Cron\KnownCrawlerRefresh::{closure:Zee\BotGuard\Cron\KnownCrawlerRefresh::run():26}()
#4 src/addons/Zee/BotGuard/Cron/KnownCrawlerRefresh.php(26): Zee\BotGuard\Service\HealthCriticalMaintenance->runTrackedTask('known_crawler_s...', Object(Closure), 'cron', Object(Zee\BotGuard\Service\CronRunBudget))
#5 src/XF/Admin/Controller/CronEntryController.php(119): Zee\BotGuard\Cron\KnownCrawlerRefresh::run(Array)
#6 src/XF/Mvc/Dispatcher.php(362): XF\Admin\Controller\CronEntryController->actionRun(Object(XF\Mvc\ParameterBag))
#7 src/XF/Mvc/Dispatcher.php(264): XF\Mvc\Dispatcher->dispatchClass('XF:CronEntry', 'Run', Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\CronEntryController), NULL)
#8 src/XF/Mvc/Dispatcher.php(121): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\CronEntryController), NULL)
#9 src/XF/Mvc/Dispatcher.php(63): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#10 src/XF/App.php(2824): XF\Mvc\Dispatcher->run()
#11 src/XF.php(814): XF\App->run()
#12 admin.php(15): XF::runApp('XF\\Admin\\App')
#13 {main}
["url"] => string(54) "/community/admin.php?cron/zeeBgKnownCrawlerRefresh/run"
["referrer"] => string(53) "https://*************/community/admin.php?cron/"
["_GET"] => array(1) {
["cron/zeeBgKnownCrawlerRefresh/run"] => string(0) ""
}
["_POST"] => array(1) {
["_xfToken"] => string(8) "********"
}
}
- RuntimeException: XF Bot Guard known crawler refresh failed: Known crawler range verification cache could not be invalidated.
- src/addons/Zee/BotGuard/Service/KnownCrawlerRefresher.php:575
- Generated by: Paul
- Jul 7, 2026 at 5:42 PM
Stack trace
#0 src/addons/Zee/BotGuard/Service/KnownCrawlerRefresher.php(220): Zee\BotGuard\Service\KnownCrawlerRefresher->assertKnownCrawlerRangeDirectoryInvalidated()#1 src/addons/Zee/BotGuard/Service/KnownCrawlerRefresher.php(39): Zee\BotGuard\Service\KnownCrawlerRefresher->seedSourcesIfNeeded()
#2 src/addons/Zee/BotGuard/Cron/KnownCrawlerRefresh.php(29): Zee\BotGuard\Service\KnownCrawlerRefresher->refreshAll(false, Object(Zee\BotGuard\Service\CronRunBudget))
#3 src/addons/Zee/BotGuard/Service/HealthCriticalMaintenance.php(240): Zee\BotGuard\Cron\KnownCrawlerRefresh::{closure:Zee\BotGuard\Cron\KnownCrawlerRefresh::run():26}()
#4 src/addons/Zee/BotGuard/Cron/KnownCrawlerRefresh.php(26): Zee\BotGuard\Service\HealthCriticalMaintenance->runTrackedTask('known_crawler_s...', Object(Closure), 'cron', Object(Zee\BotGuard\Service\CronRunBudget))
#5 src/XF/Admin/Controller/CronEntryController.php(119): Zee\BotGuard\Cron\KnownCrawlerRefresh::run(Array)
#6 src/XF/Mvc/Dispatcher.php(362): XF\Admin\Controller\CronEntryController->actionRun(Object(XF\Mvc\ParameterBag))
#7 src/XF/Mvc/Dispatcher.php(264): XF\Mvc\Dispatcher->dispatchClass('XF:CronEntry', 'Run', Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\CronEntryController), NULL)
#8 src/XF/Mvc/Dispatcher.php(121): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(XF\Admin\Controller\CronEntryController), NULL)
#9 src/XF/Mvc/Dispatcher.php(63): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#10 src/XF/App.php(2824): XF\Mvc\Dispatcher->run()
#11 src/XF.php(814): XF\App->run()
#12 admin.php(15): XF::runApp('XF\\Admin\\App')
#13 {main}
Request state
array(4) {["url"] => string(54) "/community/admin.php?cron/zeeBgKnownCrawlerRefresh/run"
["referrer"] => string(53) "https://*************/community/admin.php?cron/"
["_GET"] => array(1) {
["cron/zeeBgKnownCrawlerRefresh/run"] => string(0) ""
}
["_POST"] => array(1) {
["_xfToken"] => string(8) "********"
}
}
Amazing, love to hear the results are working out well. I am still doing work in the performance area in every single release. In my dev, each hot-path is performance sampled (phpbench) and I work to identify at least one or two material improvements inside each release. While it's impossible to achieve 0 cost, I aim to get it low and prove that origin-based bot defence is indeed viable, even on already constrained shared environments and micro vps environments etc.
It flies in the face of conventional wisdom, but, I think, the results are starting to come into that shape - and over time will land fully.
Hey there,
It seems you may have run the cron manually. The cron itself configures a lock mechanism to ensure only one process may run at a time. It is possible that the cron may have already been running, so a lock was acquired already and your run was locked out.
p.s., it is not anything to worry about - if a lock fails (truely and repeatedly) then it falls back to DB-based reads. Security is not at all diminished or broken.
Is this a one-off? Or does it happen more than that?
zeeb0t updated XF Bot Guard with a new update entry:
Minor Release: v1.3.9
Read the rest of this update entry...
Minor Release: v1.3.9
Read the rest of this update entry...
Your wish is my command... 1.3.9 introduces the functionality you both sought. Enjoy
