Unit testing XenForo addons - tutorial

Framework Documentation

Every helper is documented in DOCS.md. TIPS.md covers writing testable code and what to reach for instead of mocking XenForo.

FAQs

Q: I'm getting an error: XF\Db\Exception: Too many connections when I run my tests

A: Your tests make many queries in a short time. Either make fewer queries, or raise MySQL's max_connections on your development server. If that causes memory or performance problems, tune the server with a tool such as MySQLTuner.

Q: Every test fails with This suite sets $addonsToLoad to [...], but the application was booted without it

A: Your tests/CreatesApplication.php predates $addonsToLoad and does not pass it on, so no isolation is in effect. Delete that file and the use CreatesApplication; line in tests/TestCase.php — the framework boots XenForo itself.

Q: Tests fail with Mockery expectation counts of zero

A: Another installed addon ships its own PHPUnit and Mockery, and the two copies collide. Set $addonsToLoad to your own addon id in tests/TestCase.php.

Q: The run dies with exit 255 and a message about a readonly PHPUnit class

A: Readonly class PHPUnit\Framework\TestStatus\Known cannot extend non-readonly class, or something like it, means two copies of PHPUnit are on the class loader and a class has come from one with its parent from the other. The versions do not have to differ by a major — two patch releases of the same minor are enough. Set $addonsToLoad to your own addon id, and no other addon's autoloader is registered. Note that XenForo's error handler writes that fatal to xf_error_log as a real row, so an addon which reports on the error log will mail it onward.

Q: composer update refuses to install league/flysystem-memory for swapFs()

A: "not loaded, because they are affected by security advisories". Every flysystem release up to 3.35.2 carries a low-severity advisory and Composer blocks an affected package by default; XenForo supplies flysystem 1.x, which has no patched release, so the version you need is the version that is flagged. The installation section of the resource post has the two lines your own composer.json needs — accept that one advisory by id, and keep a league/flysystem: ^1.1.4 floor so the critical advisory that is patched stays enforced. Do not ignore PKSA-pwh8-d4fr-nywn.

Q: A reply assertion throws saying its last argument is a failure message

A: assertReplyParam($reply, $key, $expected) has no expected-value argument — the trailing argument of every reply assertion is a failure message, so the value was never compared and the assertion passed whatever the reply held. assertReplyParamSame($reply, $key, $expected) is what it meant. A grep finds only the ones written as a literal; where the value is a variable or a method call, this refusal is what finds them.

Note what is not refused: a string there cannot be told apart from a failure message and is taken as one. So assertReplyParam($reply, 'key', 'a value') still asserts no more than that the key exists, silently. Only a non-string is refused.

Q: I dumped a reply to see what it held and my database password is in the test output

A: Do not var_export() or print_r() a reply or a phrase. An error reply's errors are XF\Phrase objects, and a phrase holds the language, which holds the database adapter, which holds its own configuration — so one phrase is enough to print your database password and user in plain text, into the test output and from there into any build log that captured it. describeReply() is the safe way to look: it renders each error phrase to its text. So are replyErrors() and the reply assertions, which compare rendered text, so their own failure output is safe to paste. If you have already done it, treat the log as carrying a credential.

Q: An output assertion refuses the argument I gave it

A: You wrote a value into a slot that means something else, and it used to be accepted silently. assertSee(), assertDontSee() and assertSeeInOrder() take the escaping flag third and the failure message fourth, while assertSeeText() and assertDontSeeText() take no flag at all — so a message written third was merely truthy, leaving the assertion behaving as it does by default with your message gone. An empty value is refused too, since it is found at position 0 of anything: a positive assertion could not fail and a negative one could not pass. The refusal prints that assertion's own argument list.

Q: callAction() refuses my controller class name

A: You named your own extension of a controller. Name the class your addon extends instead — 'XF:Tools', not MyVendor\MyAddon\XF\Admin\Controller\Tools. XenForo resolves the base to the most derived class, which is yours, so passing the base is equivalent and order-independent — and resolving through it is itself proof the extension applied. Naming the extension worked only when something earlier in the run had resolved the class being extended, which is why the same call passed in a full suite and failed under --filter with Class "...\XFCP_..." not found.

Q: Cannot declare class ... XFCP_... because the name is already in use

A: A class was extended twice in one process. The XFCP proxy is a class_alias(), and PHP cannot declare that name twice. If this is intermittent and you are below 5.10.0, upgrade: the extension cache was keyed by the spelling it was asked for rather than the class XenForo would actually extend, so a class reached by more than one name was extended once per spelling. If you are adding extensions in tests, each test that needs one now adds it — and where more than one test does, extend a class no other test resolves.

Q: A test asserting that an extension is absent passes alone and fails in a full run

A: $addonsToLoad cannot remove a class extension something earlier in the process has already resolved — XenForo caches the resolution and aliases a proxy, and a proxy cannot be undeclared. So a test proving a column stays nullable for a forum without your addon exercises the extension anyway once anything else has resolved that class. Guard it with requireClassNotExtended(), which skips rather than passing.

Q: A test class is refused in setUp(), naming a class extension that will not reach it

A: That class names more addons in $addonsToLoad than its siblings do, and the list cannot add an extension to a class the run has already resolved any more than it can remove one. So those extensions reached it only when it happened to run first, and otherwise it exercised XenForo's own class with the extension's methods missing — failing somewhere inside XenForo rather than at your test. Running that class on its own still passes, which is why the refusal is about the suite rather than the class. Declare one $addonsToLoad for the whole suite, naming every addon whose class extensions or content types any test reaches. One addon is worth thinking about before you add it: booting one that vendors its own PHPUnit puts a second copy on XenForo's class loader.

Q: A dispatch ignored the REQUEST_METHOD I put in $server — now it is refused

A: The request method comes from the $method argument, which also decides whether a CSRF token is minted — so a method supplied through $server could only ever have produced a request XenForo refuses. It was being discarded and the request sent with the method the argument named, which a test asserting on a view could pass against. Pass the method as the argument. Building an XF\Http\Request yourself and swapping it into the container is a different thing and is unaffected — there the key is how the method gets set, so do not go looking for it with a grep and remove it.

Q: A test passes in a full run but fails alone with Class "..." not found

A: It extends plain PHPUnit\Framework\TestCase, so it relies on an earlier test having booted XenForo, whose autoloader loads your addon's classes. Extend Tests\TestCase instead. To find such tests, run each test class on its own:


Q: The whole run stops with Class "XF\..." not found before any test runs

A: A test file declares a class extending a XenForo or addon class at file scope. PHPUnit loads every test file before XenForo boots. Declare it as an anonymous class inside the test method, or put a named class in a file PHPUnit does not collect, such as tests/Support/, and require_once it from the test method.

The same applies to a data provider, which PHPUnit also runs while building the suite — a class constant such as OutputInterface::VERBOSITY_DEBUG fails there with Class "Symfony\Component\Console\Output\OutputInterface" not found. Return the literal value from the provider, or resolve it inside the test.

Q: A template render started failing after upgrading to v5.4

A: From v5.4, rendered templates see $xf, as they would on a page, so code the template runs through $xf now actually runs. Pass the parameters the template needs — for example renderTemplate('admin:user_edit', ['user' => \XF::visitor()]).

Q: assertNoTemplateErrors() fails naming another addon's template

A: It is describing the forum, not your addon. Template modifications are compiled into the install's shared template cache and apply to every render, whatever $addonsToLoad says — so a sibling addon's modification reading a column its own class extension declares warns on every render, for a page of yours that is faultless in a browser. Pass the prefix of the templates you own, or a list of prefixes. The blanket form is meaningful on a single-addon install.

Q: I changed a phrase to prove my phrase assertion can fail, and nothing changed

A: Two causes, and both look like the assertion not working. A render resolves phrases from the database, so editing _output/phrases/ proves nothing — change it through its XF:Phrase entity. And do that before the render under test: the language keeps every phrase it has already resolved, so a mutation after a render in the same test is not seen. Mutating the template is the easier control. One more trap: renderMacro() never reaches content placed after </xf:macro>, so a reference put there passes vacuously.

Q: assertNoUnresolvedPhrases() reports every option on my admin option page

A: You passed your addon's bare prefix, and with development mode on XenForo displays each option's id as ordinary text — so every option on that page matches, whether or not its phrase resolved. Pass the keys instead. Worth knowing why it matters beyond the noise: the ids are displayed only in development mode, so an assertion written that way passes on an install with it off and fails on one with it on, which makes the result a fact about the install rather than about your addon.

Q: A test fails with "This test's transaction did not survive"

A: Everything the test wrote before that point is now permanent in the database. The message says which of three things happened.

Something committed it — any DDL does, MySQL commits those implicitly, and $db->emptyTable() counts because it issues a TRUNCATE. On a development install the usual cause is rendering a template whose _output/ file does not match the hash in _metadata.json, which is the state an edit leaves it in: XenForo re-imports and compiles it, and that truncates the CSS cache. Load a page that renders the template, outside any test, and the watcher writes the hash. xf-dev:import does not clear it — it reads that file without writing it — and xf-dev:export would write the database over your edited file.

The code under test ended it with rollbackAll() or commit(). XenForo calls rollbackAll() on its own error paths, so running a failing job through the job manager is the common case. Test the method directly, or leave the trait off that class and undo the writes yourself.

The server rolled it back, which a deadlock does, leaving nothing behind to say so. The check stays quiet when the test has already failed, so the deadlock's own error is what you see rather than a second wrong explanation on top of it.

Q: An intermittent "Record has changed since last read in table 'xf_data_registry'"

A: Your test wrote a registry row the rest of the forum also writes, inside its transaction, and on MariaDB 11.8 or later snapshot isolation refuses the write if anything touched the row first. Upgrade to 5.7.0 or later, which stops the framework's own deferred work writing that row, and use fakesRegistry() for a row your own code writes. Call it from the setUp() of any class that creates or deletes content under a transaction, not only in a test about the registry.

Q: An intermittent deadlock naming xf_user_group_relation when tests create users

A: Saving an XF:User rewrites that user's rows in that table, and the delete that precedes the insert locks the index gap above the last row for the rest of the transaction — so two tests creating users against the same database deadlock there even on different user ids. createEntity('XF:User') reaches it as well as createUserAccount(), and fakesRegistry() does not cover it. Saving a user is the most contended write this framework makes and a transaction does not make it reliable: it is dependable against a forum nothing else is writing to, and intermittent against a shared one. Where a test needs a visitor rather than a row, actingAsMember() builds one in memory and touches nothing.

Q: makeEntity() throws saying the entity did not take a value

A: XenForo verifies many columns as they are set rather than at save time — a unique key already in use, an invalid email, a username below the minimum length — and records a failure instead of throwing. The value was simply missing before; it now fails loudly. Use setTrusted() to set one anyway.

A read-only or auto-increment column — an entity's own id, typically — is a separate refusal with the same remedy, and it names the setTrusted() call to write. That one already failed before the refusal existed, with XenForo's own message naming forceSet, which is not an option this framework surfaces.

Q: A second fakesHttp() in one test does nothing, and the first queue reports Mock queue is empty

A: Upgrade to 5.22.0 or later. Each fake used to build its own handler stack, and a Guzzle client holds the stack it was built with — so a client your addon's sub-container had already built, or a shared one your test kept in a variable, went on using the first fake's queue. The second fake installed responses nothing read, and the first queue, by then drained, answered from inside the code under test. The stack is now built once per test and its handler replaced, so a second fake reaches every client the first one covered, and switching from fakesHttp() to fakesHttpByUrl() mid-test takes effect everywhere. The one case that still cannot work is a fake installed after a client was built with no fake in place at all.

Writing testable code and other unit testing tips

Those have moved to TIPS.md, where they are kept up to date with the framework.