XF 2.2 What does rebuild master data actually do?

benFF

Well-known member
Licensed customer
I've tried upgrading my PHP version to 8.3 and it runs fine for a while, then suddenly goes all haywire and crashes the site, giving loads of "template missing" and "can't read getter" errors.

Rebuilding the master data fixes this, but that's quite a pain.

Dropping back to PHP 8.2 and there are no issues.

I'd like to try and investigate this without needing to do a rebuild, so I'd like to know what that actually does to see if there's something I can do manually first.

Cheers.
 
Courtesy of RobinGPT, some info that may point you in the right direction :) :

-------

In plain terms, Rebuild master data reinstalls XenForo’s core definition layer from the files on disk into the database, then regenerates the compiled data and caches used at runtime.

Conceptually, it is roughly:

Code:
XenForo core data files → database master records → compiled templates and caches

It rebuilds or reimports things such as:

  • Master templates and phrases
  • Style-property and template-modification definitions
  • Option and permission definitions
  • Routes and public/Admin CP navigation
  • Cron entries, widgets and content-type definitions
  • Code-event listeners and class extensions
  • BB codes and media-site definitions
  • Compiled templates, phrases and permission caches

The command-line output therefore normally contains stages such as:

Code:
Importing... Master data
Rebuilding... Phrases
Rebuilding... Templates
Rebuilding... Permissions

It can be run from the installer or from the command line:

Code:
php cmd.php xf:rebuild-master-data

It does not rebuild or remove:

  • Members
  • Threads or posts
  • Conversations
  • Attachments
  • Forums and nodes
  • The search index
  • Content counters

It also normally preserves custom templates in your own styles, translations in your own languages, selected option values and configured user-group permissions. Direct modifications to the hidden master style/language could be overwritten, however, and custom templates may subsequently be marked as outdated if their restored master versions have changed.

It only rebuilds XenForo core master data. It does not rebuild every add-on’s data, repair missing PHP files, change the database schema or perform an upgrade.

What your symptoms suggest

Assuming XenForo and all add-ons are current, PHP 8.3 should be a supported configuration. The fact that the site:

  1. Runs correctly initially
  2. Later develops numerous missing-template and unknown-getter errors
  3. Temporarily recovers after rebuilding master data
  4. Works consistently after reverting to PHP 8.2

suggests a problem with generated code or caching, rather than the actual master records spontaneously becoming damaged.

The two types of error are related:

  • “Template missing/unknown” means XenForo cannot find or load the expected compiled template.
  • “Unknown/can’t read getter” means a compiled template expects an entity property or method that is not currently available—often because the corresponding add-on class extension has not loaded.

A master-data rebuild regenerates both the templates and the associated caches, temporarily bringing them back into sync.

1. PHP 8.3 OPcache or PHP-FPM

This would be my first suspect.

A compatibility bug in XenForo itself would normally be immediate and reproducible. Working for some time and then having several unrelated components fail together is more characteristic of a stateful cache or PHP-process problem.

Changing from PHP 8.2 to PHP 8.3 can also change more than the PHP binary. Depending on the hosting configuration, it may select a different:

  • PHP-FPM pool
  • php.ini configuration
  • OPcache configuration
  • APCu, Redis or Igbinary extension
  • Process user or filesystem permission set

XenForo stores compiled PHP templates under:

Code:
internal_data/code_cache/templates/

Rebuilding master data rewrites those files. This may temporarily work around stale or corrupted OPcache entries.

The next time the problem occurs, I would restart PHP 8.3/PHP-FPM without rebuilding master data.

  • If restarting PHP immediately fixes the site, OPcache or the PHP-FPM process is strongly implicated.
  • If restarting PHP does not help, check whether the referenced compiled template files actually exist and are readable.

Compare the web-server PHP-information output for PHP 8.2 and 8.3, particularly:

Code:
opcache.enable
opcache.validate_timestamps
opcache.revalidate_freq
opcache.file_update_protection
opcache.memory_consumption
opcache.max_accelerated_files
opcache.use_cwd

Use the PHP-information page served through the website, rather than relying only on php -i, because CLI PHP may use a different configuration from PHP-FPM.

2. Compiled template files disappearing or becoming unreadable

Check whether anything periodically removes or changes files beneath:

Code:
internal_data/code_cache/

Possible causes include:

  • A hosting cleanup process
  • Security or malware-scanning software
  • A deployment or backup script
  • Incorrect ownership or permissions under the PHP 8.3 handler
  • Disk-space or inode exhaustion
  • Multiple web servers without shared internal_data storage
  • A read-only or unreliable filesystem

During the next failure, inspect one of the exact compiled-template paths shown in the stack trace and check:

Code:
df -h
df -i
find internal_data/code_cache/templates -type f | wc -l
ls -ld internal_data internal_data/code_cache

Comparing the number and modification times of compiled templates before and after the failure could be particularly revealing.

I would not manually delete the contents of internal_data/code_cache/templates. Pages requiring those templates will fail until the files are regenerated.

3. Redis, Memcached or APCu

If an external object cache is configured, PHP 8.3 may be loading a different extension or serializer version from PHP 8.2.

That could cause cached class-extension or registry data to become invalid, while a master-data rebuild temporarily repopulates it.

As a test, temporarily run PHP 8.3 without XenForo’s external object-cache configuration. If the problem stops, investigate:

  • The Redis/Memcached/APCu extension version
  • Whether PHP 8.2 and 8.3 use different serializers
  • Igbinary compatibility
  • The cache namespace
  • Whether multiple installations share the same cache

Be careful when flushing Redis or Memcached if it is shared with other websites.

4. A specific add-on

Unknown-getter errors frequently identify the responsible add-on through the getter name. For example:

Code:
Accessed unknown getter 'ozzmodz_example' on XF:User

The prefix often identifies the add-on whose template modification is calling the property.

If all the errors point towards one add-on, investigate that add-on’s PHP 8.3 compatibility. If unrelated add-ons start reporting unknown getters simultaneously, a central OPcache, object-cache or filesystem problem is much more likely than several add-ons independently failing at the same time.

Suggested diagnostic sequence

The next time it happens, avoid rebuilding immediately:

  1. Save several complete XenForo error entries, including their stack traces.
  2. Record the exact missing-template and getter names.
  3. Restart only PHP 8.3/PHP-FPM.
  4. If the problem remains, check whether the referenced compiled-template files exist and are readable.
  5. Check disk space, inodes, ownership and modification times.
  6. Compare the PHP 8.2 and 8.3 PHP-information pages and loaded extension versions.
  7. Temporarily disable the external XenForo cache, if configured.
  8. Group the unknown-getter errors by add-on prefix.
  9. Run XenForo’s file-health check.
  10. Only then rebuild master data.

Based on the delayed onset and the fact that rebuilding master data temporarily fixes several apparently unrelated errors, my leading suspicion would be the PHP 8.3 handler—particularly OPcache or a PHP 8.3-specific caching extension/configuration—causing compiled templates and class-extension caches to become inconsistent.

Restarting PHP-FPM instead of rebuilding master data is probably the single most useful first test.


Sources and further reading

The following references may be useful when investigating this further:

  • XenForo 2.3.12 release and system requirements

    Confirms that PHP 8.3 is the recommended PHP version for the current XenForo 2.3 release. This makes a general incompatibility between current XenForo core and PHP 8.3 less likely.
  • XenForo: What rebuilding master data affects

    Discussion confirming that rebuilding master data restores master templates and phrases but does not affect members, threads, posts or other forum content.
  • XenForo command-line rebuild commands

    Includes the
    Code:
    xf:rebuild-master-data command and distinguishes rebuilding XenForo core master data from rebuilding an individual add-on.
     
     [*][URL='https://xenforo.com/community/threads/2-1-5a-rebuild-master-data.172108/']XenForo developer comment on regularly rebuilding master data[/URL]
     
     A XenForo developer states that administrators should almost never need to rebuild master data regularly. If it repeatedly becomes necessary, that indicates an underlying installation, hosting, add-on or caching problem.
     
     [*][URL='https://xenforo.com/community/threads/how-to-read-and-fix-server-error-logs.218446/']XenForo explanation of internal_data and compiled templates[/URL]
     
     A XenForo developer explains that [ICODE]internal_data
    stores installation-specific non-public data, including compiled templates. The discussion also explains the language and style directory structure beneath internal_data/code_cache/templates/.
  • XenForo developer comment concerning aggressive OPcache configuration

    Provides an example where PHP had not picked up changed code and a XenForo developer suggested that an aggressive OPcache configuration could be responsible, requiring PHP to be restarted.
  • PHP manual: OPcache configuration

    Official documentation for settings including opcache.validate_timestamps, opcache.revalidate_freq, opcache.file_update_protection, opcache.memory_consumption and opcache.max_accelerated_files.
  • PHP manual: opcache_get_status()

    Documents the information available about the active in-memory opcode cache, including whether the cache is full, whether a restart is pending and the numbers of out-of-memory, hash and manual restarts.
  • PHP manual: opcache_reset()

    Explains how the in-memory opcode cache can be reset. Note that PHP CLI and the web server/PHP-FPM commonly use separate OPcache instances, so calling opcache_reset() through CLI may not reset the cache used by the website. Restarting the applicable PHP-FPM service is generally a more reliable diagnostic test.
  • XenForo discussion of unknown-getter errors

    Shows how an unknown-getter error may originate from an add-on, style, modified template, navigation entry, widget or other customisation.

These references do not establish the exact cause in this case, but they support the main diagnostic possibilities: stale OPcache entries, missing or unreadable compiled templates, inconsistent external-cache data, or an add-on template expecting a class extension that has not loaded.
 
I did wonder about the opcache, but it just seems strange it would suddenly corrupt.

Restarting the fcgi process, or even changing back to 8.2 didn't fix it.

I also wondered if it was the virus scanner playing silly buggers, but the code cache folder is excluded from scan.

I didn't actually check if the mentioned file exists though, so I'll try that next if it happens again.
 
Have you got the forum running through Cloudflare?

Just a suggestion if you have, try turning off Cloudflare caching before you make the changes and see if that helps.
If it does clear the cache data and turn CF caching back on
 
It's very peculiar as it hasn't done it again since.

So rebuild only deals with rebuilding files? It doesn't touch the database?

(I'm just trying to think what could have possibly got corrupted...)
 
So rebuild only deals with rebuilding files? It doesn't touch the database?
No, indeed the database is touched, e.g. permission rebuild, new version strings for CSS and JS files, ...

From time to time we also have to rebuild master data of a test site, because the site's layout crashed after server upgrades. We never really investigated the exact reasons for this.

AFAIK this can have something to do with caching. Look at Xon's recent Redis cache update notes:

 
No, indeed the database is touched, e.g. permission rebuild, new version strings for CSS and JS files, ...

From time to time we also have to rebuild master data of a test site, because the site's layout crashed after server upgrades. We never really investigated the exact reasons for this.

AFAIK this can have something to do with caching. Look at Xon's recent Redis cache update notes:

Ah yes, someone else in the thread had a similar issue. And I had just switched to redis 🤔
 
Back
Top Bottom