[XTR] IP Threat Monitor 1.0.28

• Expanded error detection (401, 403, 404 HTTP codes)
• Login redirects now counted as errors (for guest users)
• Added error template detection
• Separate counter system working properly

This update fixes a critical bug in the error page rate limiting feature based on customer feedback. The feature now works as intended, providing better protection against scraping bots that ignore error responses!

Scraping bots often ignore error responses and continue hammering your site with requests to restricted pages, user profiles, and non-existent content. This fix ensures these bots are now properly detected and blocked, reducing server load and protecting your content.

---

Configuration

Location: Admin CP → Options → IP Threat Monitor

• Enable Error Page Rate Limiting - Turn the feature on/off
• Error Page Threshold - Number of errors before blocking (default: 10)
• Error Page Time Window - Time period in seconds (default: 300)

Recommended Settings:

• Balanced: Threshold 10-15, Window 300 seconds
• Aggressive: Threshold 5-10, Window 180 seconds

Bug Fixes

ASN Comment Parsing Bug Fixed
• Comment usage (#) in ASN lists now works properly
• "AS12345 # DigitalOcean" format is supported
• Comments are parsed during input, only clean ASN numbers are stored
• Implemented following XenForo CensorWords pattern

New Features

Error Page Rate Limiting
• Separate rate limiting for IPs repeatedly hitting 403/permission errors
• Addresses scraping bots that ignore error pages
• Separate thresholds and time windows
• Auto-ban integration

Manual IP Blacklist
• Manual IP blacklist system like trusted IPs
• IP address + comment support
• CIDR notation support (192.168.1.0/24)
• Instant blocking (on first visit)

Country Blocking Toggle
• Option to temporarily disable country blocking
• Test without deleting country list
• Single checkbox toggle

VPN Provider Whitelist
• Whitelist specific VPN providers
• Protect legitimate VPNs like Opera VPN, ProtonVPN
• Provider name + comment support
• Flexible matching system (exact, contains, partial)

Search Engine Bot Configuration
• Select which bots to protect
• 18 different bot support (Google, Bing, Yandex, Baidu, etc.)
• Easy checkbox selection
• Protect critical bots for SEO

Improvements

Test Configuration Updates
• Added test support for all new features
• Smart test adaptation based on Setup Wizard presets
• Optimized test criteria for Starter preset
• 6 new test scenarios

Setup Wizard Integration
• All presets updated with new features
• Starter: Simple setup, essential bots
• Standard: Balanced protection, comprehensive bot list
• Aggressive: Maximum security, all bots

Diagnostics Page Updates
• Bot protection status check
• Protected bot count and list display
• SEO warnings and recommendations
• Automatic setting links

Technical Improvements

• XenForo AbstractOption pattern usage
• getCheckboxRow method for checkbox rendering
• Proper validation and error handling
• Cache-first approach optimizations
• Template syntax improvements
• Extended phrase system (50+ new phrases)

Important Notes

• All new features are backward compatible
• Existing settings are preserved
• Setup Wizard recommended for quick configuration
• Test Configuration recommended for validation

This major update brings 6 significant new features and improvements based on customer feedback. We now offer a more flexible, powerful, and user-friendly IP protection experience!

Highlighted Features

Smart Bot Management
Now you have full control over which search engine bots are protected! Want to protect Google and Bing while excluding Yandex? Now it's possible!

• 18 different bot options (Google, Bing, Facebook, Twitter, etc.)
• Easy checkbox selection
• Protect critical bots for SEO
• Exclude unused bots

Advanced Security Controls
Scraping bots can no longer ignore error pages! The new error page rate limiting system catches IPs that repeatedly hit 403 errors.

• Separate error page thresholds
• Specialized protection for 403/permission errors
• Auto-ban integration
• Scraping bot protection

Flexible Configuration Options
Full control with manual IP blacklist and VPN whitelist!

• Manual IP blocking (with CIDR support)
• VPN provider whitelist
• Temporary country blocking toggle
• Comment support for organized management

User Experience Improvements

Setup Wizard Updates
• All new features integrated into 3 presets
• Starter: Simple setup for beginners
• Standard: Recommended balanced settings for most sites
• Aggressive: For sites requiring high security

Test Configuration Smart Adaptation
• Tests now adapt based on your selected preset
• No unnecessary warnings if you chose Starter preset
• Optimized test criteria for each preset

Diagnostics Page Expanded
• Bot protection status display
• List of protected bots
• SEO warnings and recommendations
• One-click navigation to settings pages

Why Should You Update?

1. Better SEO Control: Optimize your SEO by selecting which bots to protect
2. Enhanced Security: Additional protection against error page attacks
3. Easy Management: Instantly block problem IPs with manual IP list
4. VPN Flexibility: Protect legitimate VPN users
5. Smart Testing: Test system adapted to your configuration

Update Recommendations

• Run Setup Wizard after update
• Verify your settings with Test Configuration
• Review your bot protection settings
• Visit Diagnostics page to explore new features

• New: Completely rewrote the Blacklist query architecture to fix slowness on large databases (O(1) Optimization).
• New: Added "Blacklisted IP Retention" (Auto-Prune) option to prevent database bloating.
• Improvement: Integrated the new retention setting into Setup Wizard profiles.
• Improvement: Added database retention check to the Test Configuration tool.
• Fix: Added missing phrases.

This update contains a critical performance patch, especially for forums with high traffic and hundreds of thousands of "Blacklisted" records in their database.

Highlights:

• Blazing Fast Queries: The IP check mechanism, which could previously lock up the server, now responds in milliseconds even with millions of records in the database.
• Database Hygiene: You can now automatically prune blacklisted IP records older than a specified number of days (e.g., 30 days). This feature smartly deletes only unnecessary data without touching active bans or the whitelist.

We strongly recommend all customers upgrade to this version.

• [NEW] Integrated MaxMind GeoIP2 library. Country lookups are now performed via a local database.
• [NEW] Added a Setup Wizard with 3 pre-defined protection profiles (Starter, Standard, Aggressive).
• [NEW] Added a Cron Job system to automatically download and update the GeoIP database.
• [NEW] Added "Composer Autoload" support (Essential for loading Vendor libraries correctly).
• [IMPROVEMENT] Reduced dependency on 3rd party APIs, resulting in significant performance gains.
• [FIX] Resolved GeoIP class loading issues on certain server configurations.

With this release, we have completely overhauled the infrastructure and are excited to announce Local MaxMind GeoIP2 Integration, eliminating the dependency on external APIs!

Key Features

• MaxMind GeoIP2 Integration (NEW):You are no longer dependent on external services or limited APIs (like proxycheck.io) for IP country lookups!
  • Unlimited and lightning-fast lookups using a local database hosted on your server.
  • New Scheduled Task (Cron job) to automatically update the GeoIP database.
  • Analyze your traffic without worrying about API limits.
• Smart Setup Wizard (NEW):No need to struggle with complex settings. Configure the add-on with a single click using our new wizard:
  • Starter: Logging only, no blocking.
  • Standard: Balanced protection, shows Captcha for suspicious activity.
  • Aggressive: High security, ideal if you are under attack.
  After updating, it is recommended to manually run the "XENTR: Update GeoIP Database" task once from Admin Log > Tools > Cron entries.

Bug Fixes

• Fixed Apple Private Relay memory exhaustion error
  Apple's iCloud Private Relay IP list has grown to ~287,000 entries, causing "Allowed memory size exhausted" errors on high-traffic sites.
• Implemented stream-based CSV parsing
  Instead of loading the entire IP list into memory at once, the file is now read line-by-line in 4KB chunks.
• Added IPv6 prefix deduplication
  245,000+ IPv6 /64 entries are now aggregated into ~25,000 unique /48 prefixes.
• Added IPv4 CIDR filtering
  Only /28 and larger subnets are retained, eliminating unnecessary granular entries.
• Added thundering herd protection (Lock mechanism)
  Prevents concurrent requests from simultaneously hitting the Apple API when cache is empty.

Performance Improvements

• Memory usage: 100% reduction (32 MB → 0 B increase)
• List size: 90% reduction (287K → 27K entries)
• Cache file: 85% smaller (~15 MB → ~2 MB)

This update addresses a critical memory exhaustion error caused by the excessive growth of Apple's iCloud Private Relay IP list.

What Was the Problem?​

Apple's official Private Relay IP list (egress-ip-ranges.csv) has grown to approximately 287,000 entries. The previous version loaded this entire list into memory at once, causing:

• On high-traffic sites
• With 256 MB or lower PHP memory limits
• During cache-cold moments (after server restart)

The error "Fatal Error: Allowed memory size exhausted" to occur.

What Changed?​

1. Smart Streaming: The IP list is now read in 4KB chunks instead of being loaded entirely into memory.
2. IPv6 Optimization: 245,000 IPv6 addresses are now deduplicated into unique /48 prefixes (~25,000).
3. Lock Mechanism: Prevents multiple concurrent requests from hitting the Apple API simultaneously.

Results​

Metric	Before	After
Memory Increase	32+ MB	0 MB
List Size	287,000	27,000

Update Recommendation​

All users who have enabled Apple Private Relay exemption are strongly encouraged to update to this version.

After updating, toggling the "Allow iCloud Private Relay" option off and on in the add-on settings will regenerate the cache.

• Feature:Added Force API Check for Block Lists option. Ensure immediate blocking of visitors from banned Countries/ASNs by forcing an API lookup on their first visit, regardless of the VPN check mode.
• Bug Fix: Addressed a logical issue in "Top Threat Countries" dashboard stats where legitimate visitors were incorrectly included in the count. It now strictly reflects Blocked and Blacklisted IPs.
• Bug Fix: Fixed a critical "Call to a member function setValue() on null" error in IPThreatLog repository which could crash the system during API health checks or flag retrieval.
• Bug Fix:Resolved an issue where comments in the Blocked ASN list (text after #) were being stripped upon save.
• Visual: Fixed missing country flags in the admin dashboard IP list by ensuring country data is properly populated via the new Force Check mechanism.

This maintenance and feature update addresses specific customer feedback regarding immediate blocking and statistics accuracy.

• Instant Country/ASN Blocking (Force Check): We've introduced a new option: "Force API Check for Block Lists". Previously, the system might not immediately check visitors if they weren't suspicious, meaning someone from a blocked country could browse as a guest. With this new option enabled, every new visitor is checked against the API immediately. If they match your Blocked Country or ASN list, they are blocked on their very first request. (Note: Enabling this will increase API usage).
• Dashboard Statistics Fix: The "Top Threat Countries" widget was previously counting all visitors. It has been updated to count only Blocked and Blacklisted IPs, providing a true picture of your threat landscape.
• Critical Stability Fix: Fixed a server-side error related to caching logic that could occur during API communication.
• Usability:
  • Comments in your ASN Block list are now preserved correctly.
  • Fixed missing country flags in the IP Log list (flags will appear as new data is populated via Force Check).

• Rewrote the file locking mechanism to use Non-blocking I/O. This prevents server hangs/lag during heavy DDOS attacks or high traffic.
• Added a new option to disable admin alert notifications to prevent inbox flooding during attacks.
• Fixed a fatal error that could occur when uninstalling the add-on.
• Fixed an issue where saving settings (Blocked ASNs, Countries, etc.) would fail with invalid input. Invalid entries are now gracefully stripped.
• Added automatic database schema repair logic during upgrade. If tables or columns are missing, they will be recreated automatically.
• Fixed a character encoding issue affecting the German language pack.

This update brings critical improvements regarding stability and performance. We have optimized processes that could impact server load during high-traffic attacks and resolved issues related to the installation/uninstallation process.

Highlights:

• Smarter & Faster: The blocking mechanism now operates in a non-blocking mode, ensuring server resources are used much more efficiently without causing delays.
• Better Management: Admin alert notifications sent during attacks can now be disabled via options.
• Robust Installation: The updater now includes a self-healing feature that automatically repairs missing or corrupted database tables during the upgrade.

• Fixed: Resolved the critical "MySQL Data too long" error caused by large IP lists exceeding database capacity.
• Fixed: Database performance issues caused by frequent updates from the Rate Limiter and IP services.
• Changed: The caching system has been completely rebuilt to use file-based storage instead of the database, ensuring zero database load for large datasets.
• Improvement: Optimized cache cleaning processes for better stability.

Critical Fixes & Improvements

This update introduces a major architectural change to the caching mechanism, migrating from database-based storage to a dedicated file-based system to permanently resolve database size limits.

• ixed "Data too long for column 'data_value'" database error occurring when caching the Cloudflare IP list.
• Migrated Cloudflare IP caching mechanism from database (SimpleCache) to file-based system (File System).
• Fixed "Accessed unknown getter 'xentr_ipt_country'" error when updating user country information.

This update resolves two critical issues frequently appearing in Server Error Logs.

1. SimpleCache Size Limit Issue: XenForo's default caching mechanism (SimpleCache) stores data in the database with a specific size limit. When combined with data from other installed add-ons, the Cloudflare IP list could exceed this limit, triggering "Data too long" errors. Cloudflare IP data is now securely stored in your server's file system (internal_data folder).
2. Entity Getter Error: In some cases, an "Accessed unknown getter" error was resolved when updating user country information.

• Fixed "Allowed memory size exhausted" error when accessing Dashboard and Analytics pages on databases with large amounts of records.
• Chart and statistics data processing has been switched to SQL-based aggregation to prevent PHP memory exhaustion.
• Performance improvements made to statistical queries.

This update includes a critical performance patch, especially for forums with high traffic and tens of thousands of IP log records stored in the database.

In previous versions, loading all data into server memory (RAM) to generate charts on the Dashboard and Analytics pages could cause memory limit exhaustion (Fatal Error) in some cases. With version 1.0.19, this process has been optimized by offloading the workload to the database engine. This ensures your Dashboard page loads fast and without errors, even with millions of log records.