XF Bot Guard

XF Bot Guard 1.4.1

No permission to download
Overview FAQ Updates (41) Reviews (20) History Discussion
duderuud said:
Upgraded to latest version and ran a manual cron update for the MMDB update.
Code: 
The downloaded local IPLocate country or ASN database could not be validated. Bot Guard will ignore unavailable local geo lookup data, continue to fail open, and retry through the MMDB refresh cron.

Still not working. I checked the status.json:
Code: 
    "status": "error",
    "last_checked": 1782389629,
    "last_success": 0,
    "last_error": "Downloaded MMDB file did not validate: reader_unavailable",
    "manifest_etag": "",
    "manifest_last_modified": "",
    "manifest_generated_at": "",
    "not_modified": false,
    "files": []

Could it be that a CF or security plugin is the culprit?
Click to expand...

Well, this is progress. The db files downloaded, but now, it won't load the MaxMind DB reader class.

Please run the XenForo file check - let's make sure the add-on is not missing files, and then:

Do you have another add-on using the MaxMind\Db\Reader (aka, some other add-on converting IP to country / ASN or something similar)?
 
Ridemonkey said:
FWIW, I couldn't find a corresponding entry in any of the admin navigation tables, nor could I find it in development mode, and it didn't seem to be coming from my cache. Initiating a master data rebuild from the CLI removed the offending entry.
Click to expand...

Thanks, a fix for this is coming in the next release. It will rebuild the cache for the menu on uninstall.
 
XF Bot Guard Compatibility Report: XenForo 2.3.11

XF Bot Guard has been reviewed against XenForo 2.3.11 and is compatible.

No upgrade-blocking issues were found. No XF Bot Guard code changes are required before upgrading to XenForo 2.3.11.

Compatibility Review

The XenForo 2.3.11 source was reviewed against XF Bot Guard’s XenForo integration points, including:

  • class extensions
  • code event listeners
  • template modifications
  • public request handling
  • online and session handling
  • crawler handling
  • dashboard and event behaviour
  • JavaScript behaviour
  • cross-version compatibility expectations

The existing XF Bot Guard test suite passes against XenForo 2.3.11.

Notable XenForo 2.3.11 Change

XenForo 2.3.11 expands its handling of browser prefetch and prerender requests, including requests using headers such as Sec-Purpose: prefetch or Sec-Purpose: prerender.

XenForo uses this handling mainly to avoid side effects from speculative browser requests, such as updating view counts, thread-read state, alert-read state, or session activity. XenForo still allows the page body to render.

This does not create a current XF Bot Guard compatibility issue.

XF Bot Guard does not treat these headers as a bypass, trusted signal, or validation substitute. Protected traffic remains protected, and the current behaviour is safe for upgrade.

Future Hardening Opportunity

A future XF Bot Guard patch will refine how protected, untrusted prefetch and prerender requests are handled.

For speculative browser requests, XF Bot Guard should ideally avoid returning an interactive browser-validation or CAPTCHA flow when validation would otherwise be required. Instead, it should return a bounded non-interactive response while continuing to withhold protected body content.

In plain terms: XF Bot Guard already protects these requests today. The planned improvement is to make the response shape better suited to non-interactive browser prefetch and prerender behaviour.

Recommendation

Customers can safely upgrade to XenForo 2.3.11.

A follow-up XF Bot Guard patch will refine prefetch and prerender handling, but this is a product-hardening improvement and is not required as an upgrade blocker.
 
zeeb0t said:
If your server is almost at its absolute capacity, you cannot run Bot Guard in full mode. Please wait until the no-db mode is available some time next week.
Click to expand...

Can you mention me please when this is done.
 
zeeb0t said:
Do you have another add-on using the MaxMind\Db\Reader (aka, some other add-on converting IP to country / ASN or something similar)?
Click to expand...
I think I have 3 programs downloading the daily mmdb file. Andy has a nice one that grabs it once daily, for all of his programs, to a common folder. /home/ACCOUNTID/maxmind However, these other programs only grab GeoLite2-City.mmdb
 
Rhody said:
I think I have 3 programs downloading the daily mmdb file. Andy has a nice one that grabs it once daily, for all of his programs, to a common folder. /home/ACCOUNTID/maxmind However, these other programs only grab GeoLite2-City.mmdb
Click to expand...

Thanks - that makes sense.

At the moment, Bot Guard is deliberately strict about the MaxMind reader it uses. It ensures the reader is the bundled/expected version so I can rely on the capabilities and behaviour it needs. Similarly, the databases it uses.

That said, I am making a change so Bot Guard can accept an already-loaded MaxMind reader where another add-on has loaded one first - but only where it is materially compatible, including minimum version and required capabilities. If it does not meet that bar, Bot Guard will fall back to protecting itself and not load the MaxMind DB, rather than assuming compatibility, and potentially creating weakness in defensive posture.

Apt Irrelevance said:
Perhaps there could be an option to specify a directory for the files rather than attempting to download.
Click to expand...

I will think about that.

The reason I am cautious is that Bot Guard is a security product, not just a convenience utility. The GeoIP database and reader are part of a key detection capability, so I need to be defensive about which reader is used, which database is used, and whether the data is fresh enough to rely on.

I understand the appeal of pointing multiple add-ons to a shared directory, and I do want Bot Guard to be practical. But I also need to avoid situations where another add-on, stale database, mismatched file, or unexpected setup causes false confidence or weakens protection.

So I am not ruling it out, but if I add that kind of option it would need proper validation and safeguards rather than simply trusting any file in a chosen directory.
 
Our file check job has failed two days in a row. The stack trace references bot guard in both errors:

Code: 
Exception: Job XF\Job\FileCheck: Allowed memory size of 134217728 bytes exhausted

Stack trace

#0 [internal function]: XF\Job\Manager->handleShutdown()
#1 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(38) "https://boards.theforce.net/bot-guard/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
 
Apt Irrelevance said:
Our file check job has failed two days in a row. The stack trace references bot guard in both errors:

Code: 
Exception: Job XF\Job\FileCheck: Allowed memory size of 134217728 bytes exhausted

Stack trace

#0 [internal function]: XF\Job\Manager->handleShutdown()
#1 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(38) "https://boards.theforce.net/bot-guard/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
Click to expand...

This stack trace is not showing Bot Guard itself failing.

The failing job is XenForo’s own File Health Check job:

XF\Job\FileCheck

The /bot-guard/ URL in the request state is just the referrer/page that happened to trigger XenForo’s background job runner. If your site is using XenForo’s activity-based job trigger, then normal front-end requests can trigger job.php to process pending jobs. With Bot Guard active, a lot of bot traffic will naturally be sitting on the Bot Guard challenge page, so it is not unusual for that page to appear as the referrer.

In other words: the visitor/request was on the Bot Guard page when XenForo ran a pending FileCheck job. That does not mean Bot Guard is the job consuming the memory.

I’d recommend raising it with XenForo: https://xenforo.com/community/forums/bugs/
 
Activity-based job trigger? Pretty sure this is just the standard daily filecheck cron. The errors occurred at the same time both days, the same minute the daily cron runs.

Both days two memory errors triggered. This is the other one:
Code: 
ErrorException: Fatal Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 77869200 bytes) src/XF/Util/Hash.php:14
Generated by: Unknown account Jun 25, 2026 at 6:10 PM

Stack trace

#0 [internal function]: XF::handleFatalError()
#1 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(38) "https://boards.theforce.net/bot-guard/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
 
Apt Irrelevance said:
Activity-based job trigger? Pretty sure this is just the standard daily filecheck cron. The errors occurred at the same time both days, the same minute the daily cron runs.

Both days two memory errors triggered. This is the other one:
Code: 
ErrorException: Fatal Error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 77869200 bytes) src/XF/Util/Hash.php:14
Generated by: Unknown account Jun 25, 2026 at 6:10 PM

Stack trace

#0 [internal function]: XF::handleFatalError()
#1 {main}

Request state

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(38) "https://boards.theforce.net/bot-guard/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}
Click to expand...

The second error points to something more specific. Older XF Bot Guard builds shipped the IPLocate MMDB files inside the add-on directory, and those files were also listed in hashes.json. One of them, ip-to-asn.mmdb, is around 78 MB. XenForo’s file health check hashes files by loading the whole file into memory, and your error says PHP tried to allocate about 77.9 MB at XF\Util\Hash.php:14. That lines up very closely.

So the likely issue is:

  • XenForo’s own File health check job is failing.
  • It is probably failing while checking a large file that was shipped by an older XF Bot Guard package.
  • This is especially likely if your installed Bot Guard version is 1.3.3 or 1.3.4, or if src/addons/Zee/BotGuard/hashes.json still lists data/iplocate/ip-to-asn.mmdb or data/iplocate/ip-to-country.mmdb.

Please check your installed XF Bot Guard version and the contents of src/addons/Zee/BotGuard/hashes.json. If those MMDB paths are listed, upgrade XF Bot Guard to 1.3.5 or newer. Current builds no longer ship or hash those large MMDB files; the downloaded databases are managed under internal_data instead.

As a short-term workaround, increasing the PHP memory_limit may allow XenForo’s file check to complete, but the cleaner fix is to update Bot Guard so those large runtime database files are no longer part of the add-on file health check.
 
@Anthony Parsons I saw your post, but it's vanished now. Nevertheless, I think it created some value.

You’re right that the current counter path seems heavier than it needs to be.

That said, the counters intentionally run even when audit rows are suppressed, because those are operational counters used for scoring, rate limits, Dashboard visibility and diagnostics.

Still, your performance point is valid. I’ve validated a product-neutral optimisation: remove the per-counter LAST_INSERT_ID() readback/current-bucket cache write, then batch all counter increments for an event into a single multi-row upsert. That keeps the same counters and decisions, but turns ordinary events from multiple upserts/readbacks into one query, and bootstrap/fingerprint paths from up to 4-5 counter operations into one query.

Thank you!
 
this addon has become a bit too complicated for me lol.

so i have these two warnings. i have ran the related crons manually but that does not seem to help.

The downloaded local IPLocate country or ASN database could not be validated. Bot Guard will ignore unavailable local geo lookup data, continue to fail open, and retry through the MMDB refresh cron.
Click to expand...

Recent resolver errors are present. Requests fail open, and resolver errors are cached only briefly.
Click to expand...

there are a few others but they appear to be related to database activity and time set for pruning and stuff.
 
Chromaniac said:
so i have these two warnings. i have ran the related crons manually but that does not seem to help.
Click to expand...

I just tried a fresh install and didn't have the issue, once the cron ran.

Can you send me the output of the following queries, please?

SQL: 
SELECT addon_id, title, version_id, version_string, active
FROM xf_addon
WHERE addon_id = 'Zee/BotGuard';

SELECT entry_id, active, next_run, FROM_UNIXTIME(next_run) AS next_run_time,
       cron_class, cron_method, run_rules, addon_id
FROM xf_cron_entry
WHERE entry_id = 'zeeBgMmdbRefresh';

SELECT error_id, FROM_UNIXTIME(exception_date) AS error_time,
       exception_type, message, filename, line
FROM xf_error_log
WHERE message LIKE '%MMDB%'
   OR message LIKE '%Bot Guard MMDB%'
   OR message LIKE '%thestockexchange%'
   OR message LIKE '%ip-to-country%'
   OR message LIKE '%ip-to-asn%'
ORDER BY exception_date DESC
LIMIT 20;

And from the XenForo web root (terminal commands):

Bash: 
ls -la internal_data/zee/botguard/mmdb
cat internal_data/zee/botguard/mmdb/status.json
date -u
php -r 'echo PHP_VERSION, " ", time(), " ", gmdate("c"), PHP_EOL;'

php -r '$dir=__DIR__; require $dir . "/src/XF.php"; XF::start($dir); $m=new \Zee\BotGuard\Service\MmdbResourceManager(); echo json_encode($m->healthStatus(), JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES), PHP_EOL;'
 
Anthony Parsons said:
Sorry, I deleted it, as you got a bit defensive last time I spoke about optimisation, so I removed my thoughts.
Click to expand...

Apologies, didn't mean to be defensive. Honestly, what you had originally wrote came across to me as "have you perfected everything to the maximum possible" - which is not a reality of any software on the planet. So yeah, I did feel a bit "well, no". But I didn't mean to direct that frustration at you personally. Apologies.

Don't let the zee"b0t" name fool you... I am human, and had have had a sick family the last week or two. I need a good nights sleep :P

Nevertheless your thoughts are important and helpful. So, I did catch your findings and there was a good nugget in there. I appreciate it - and have implemented on my end (just doing testing before it can be released along with some others).
 
Last edited:
Aaaand we're slowing back down again as the bot number approaches 60k. What a rollercoaster of a morning, lol.

poe-dameron-heavy-sigh.gif


EDIT: And we're back down under 1k again. Rollercoaster indeed.
 
Last edited:
You must log in or register to reply here.

Similar threads

zeeb0t
Gauging interest: optional paid bot access using x402 / Bot Guard
Replies
15
Views
270
akok
akok
Back
Top Bottom