Beta - Seeder: test data generator

Compatible XF 2.x versions

Additional requirements: PHP 8.1 or later

License: MIT

Visible branding: No

Seeder fills a development forum with realistic test content, and removes every row of it again.

It is for add-on developers who need a forum with members, dated threads, conversations, resources and the rest to test against. It is not for a live forum.

This is a beta

The classes your own seeders and factories build on may still change before 1.0. Their namespace will not.
It has been run beside a few dozen add-ons, not beside yours. Seeder switches off XenForo's own mail, alerts, push notifications, webhooks and moderator log for everything it creates. An add-on that sends something of its own when a member or post is saved is not switched off by that. Try a small run on a forum that cannot reach the outside first.

Requirements

XenForo 2.3.0 or later
PHP 8.1 or later

What it creates

Members in the states a forum holds them in: valid, awaiting approval, unconfirmed, rejected, banned and bouncing. Also members with a connected account, two-step verification, a passkey or no password, members who have logged in and never posted, and administrators with the administrator record XenForo keeps. Members can register through XenForo's own registration service, so what another add-on does when a member registers happens for them too.
Members' histories: an IP log of registering, logins and changes to the account, from addresses some members share; changes to their account in XenForo's change log, by themselves and by an administrator; earlier names, with a later request awaiting approval or rejected.
Permissions: members who hold permissions through a user group made to grant them, and groups with their own permissions on a forum, so members fall into several permission combinations and see different content.
Forums, threads and replies whose first posts cover the range of BB code, with prefixes, tags, image attachments and reactions. Threads can be sticky, featured, awaiting approval or soft-deleted, and can have a poll with votes - from every member given, where a check needs to know exactly who voted.
Forums of each type: discussion, article, question and suggestion, with threads of that type. Also search forums that list the threads of the forums they are given.
A forum tree: categories, forums inside them and forums inside those, with threads in every one.
Conversations between members, with replies at their own dates.
Profile posts one member leaves on another's profile, status updates, and comments on both.
Custom user fields with members' values, user upgrades held, run out and offered for sale (though never buyable), warnings and warning definitions, and API keys.
Widgets with a title, in any widget position.
Who is online: tens of thousands of members, guests and robots in seconds.
Logs with history: the server error log, with recurring faults from page views, commands and jobs; the admin log, with the repeats a real one has; the moderator log, in XenForo's own words, about members' threads, replies and accounts; and the job queue, with weeks of failed jobs and jobs waiting to run.
XenForo Resource Manager resources with ratings, reviews, featured resources and discussion threads, and XenForo Media Gallery items with real images and comments, where those add-ons are installed. Seeder runs without either.

Content is dated across a span of time you choose, not all at the moment it was made. A run given a seed produces the same names and text again.

What it will not do

Nothing runs until you allow it in src/config.php. The switch is there and not on the options page, so a copy of another forum's database cannot bring it along.
Nothing a seeded member or post does leaves the forum. No mail, alerts, push notifications or webhooks, and saving seeded content writes no moderator log entry. Every generated member has an address at a reserved domain such as example.com, and every seeded IP address is from the ranges set aside for documentation.
Seeded log entries are never new. None is dated in the last two hours, so whatever reports new entries does not report these. Moderator log entries are written straight into the table, so an add-on that reacts to a moderator action as it is taken does not see them. No seeded failed job is from the last day.
No seeded job ever runs. Each is due years from now and names a job class that does not exist.
It cannot be uninstalled while seeded content is recorded. Uninstalling would leave that content on the forum with nothing left that knows which it is. The refusal tells you what to run first.

Installation

Install it as you would any add-on, then allow seeding in src/config.php:

PHP:

$config['hampelSeeder']['enabled'] = true;

Then check the forum can be seeded safely:

Bash:

php cmd.php seeder:validate

That seeds a little of everything, removes it, and compares the row count of every table before and after. It is the proof that cleaning leaves nothing behind on your forum, with your add-ons installed.

It is not a read-only check. It creates content on the forum and removes it again. seeder:validate --unattended skips that part and writes nothing, as does seeder:config.

Usage

Bash:

php cmd.php seeder:run -p users=50 -p threads=100 --span=6m

That creates 50 members, two forums and 100 threads with replies, dated across the last six months. It prints what it made as JSON: every id, each member's name and address, and the one password they all share.

Everything a run creates is recorded as a batch:

Bash:

php cmd.php seeder:list            # the batches on this forum
php cmd.php seeder:clean [batch]   # remove one batch, or every batch
php cmd.php seeder:forget [batch]  # keep the content, discard the record of it

Other seeders are run by name:

Bash:

php cmd.php seeder:run User -p users=20 -p moderated=5 -p banned=4 -p admins=1 -p ips=1
php cmd.php seeder:run User -p changes=5 -p renamed=3 -p rejected=2 -p permitted=3 -p permissions=general.bypassFloodCheck
php cmd.php seeder:run Thread -p threads=30 -p reactions=3 -p tags=5 -p polls=2 -p sticky=1 -p permission_groups=4
php cmd.php seeder:run Default -p categories=3 -p forums=6 -p subforums=2
php cmd.php seeder:run ModeratorLog -p entries=200
php cmd.php seeder:run AdminLog -p entries=500
php cmd.php seeder:run Job -p failed=500 -p days=21
php cmd.php seeder:run Conversation -p conversations=10
php cmd.php seeder:run ProfilePost -p posts=8 -p statuses=3 -p comments=2
php cmd.php seeder:run Resource -p resources=6 -p reviews=2 -p featured=1
php cmd.php seeder:run Widget -p widgets=3 -p position=forum_list_sidebar
php cmd.php seeder:run Online -p guests=5000 -p robots=70000

The README lists every seeder and its parameters.

Your own seeders and factories

Seeders and factories are modelled on Laravel's, with Faker for the names and text. A factory says what one piece of content looks like by default. A seeder is a recipe that uses factories to build a whole set, and takes parameters from the command line.

The examples below are for an imaginary add-on, Acme/Tickets, which adds support tickets with replies.

Use the built-in factories to set up a case you need to test

Most testing needs no factory of your own. A seeder in your add-on's Seed directory can compose the built-in ones into exactly the forum a feature needs. This one makes a forum for testing a moderation tool: a queue with an order, some banned members, and threads in every state.

PHP:

namespace Acme\Tickets\Seed;

use Hampel\Seeder\Factory\PostFactory;
use Hampel\Seeder\Factory\ThreadFactory;
use Hampel\Seeder\Factory\UserFactory;
use Hampel\Seeder\Seed\AbstractSeeder;

class ModerationSeeder extends AbstractSeeder
{
    public function run()
    {
        $members = UserFactory::new()->count($this->count('members', 10))->create();

        // each is in the approval queue at the date they registered
        UserFactory::new()->moderated()->count($this->count('queued', 5))->create();

        // one banned for good, one for a fortnight
        UserFactory::new()->banned()->create();
        UserFactory::new()->banned(time() + 14 * 86400)->create();

        // of every four threads, one awaits approval and one was deleted with a reason
        ThreadFactory::new()
            ->recycle($members)
            ->sequence(
                [],
                [],
                ['discussion_state' => 'moderated'],
                ['deleted' => true, 'delete_reason' => 'Spam']
            )
            ->has(PostFactory::new()->count([0, 6]))
            ->count($this->count('threads', 20))
            ->create();
    }
}

Bash:

php cmd.php seeder:run Acme/Tickets:Moderation -p queued=12 --batch=moderation
php cmd.php seeder:clean moderation

recycle($members) makes every thread and reply come from those ten members. Without it each thread would create an author of its own.

A factory for your own content

A factory names your entity and gives each column a default. A default can be a value, a closure given the other attributes, or another factory, which is created when needed and replaced by its id.

PHP:

namespace Acme\Tickets\Factory;

use Hampel\Seeder\Factory\AbstractFactory;
use Hampel\Seeder\Factory\UserFactory;
use XF\Mvc\Entity\Entity;

class TicketFactory extends AbstractFactory
{
    protected $entity = 'Acme\Tickets:Ticket';

    public function definition()
    {
        $context = $this->context();

        return [
            'user_id' => UserFactory::new(),
            'subject' => rtrim($context->faker->sentence(6), '.'),
            'priority' => 'normal',
            'status' => 'open',
            // opened some time after the member joined, and never in the future
            'open_date' => function (array $attributes) use ($context)
            {
                $member = \XF::em()->find('XF:User', $attributes['user_id']);

                return $context->timeline->after($member->register_date);
            },
        ];
    }

    public function urgent()
    {
        return $this->state(['priority' => 'urgent']);
    }

    public function closed()
    {
        return $this->state(function (array $attributes)
        {
            return ['status' => 'closed', 'close_date' => $this->context()->timeline->after($attributes['open_date'])];
        });
    }

    // XenForo's clock is moved here while the ticket is saved, so anything
    // your entity stamps with the current time gets the ticket's own date
    protected function clockFor(Entity $entity): ?int
    {
        return $entity->open_date;
    }

    // extra fields for the JSON a run prints
    public static function describe(Entity $entity): array
    {
        return ['subject' => $entity->subject, 'status' => $entity->status];
    }
}

Replies belong to a ticket, so their factory names the ticket factory. for() and has() then supply a real ticket in its place.

PHP:

class TicketReplyFactory extends AbstractFactory
{
    protected $entity = 'Acme\Tickets:Reply';

    public function definition()
    {
        $context = $this->context();

        return [
            'ticket_id' => TicketFactory::new(),
            'user_id' => UserFactory::new(),
            'message' => $context->faker->paragraph(3),
            'reply_date' => function (array $attributes) use ($context)
            {
                $ticket = \XF::em()->find('Acme\Tickets:Ticket', $attributes['ticket_id']);

                return $context->timeline->after($ticket->open_date);
            },
        ];
    }
}

A seeder that builds a believable set

PHP:

namespace Acme\Tickets\Seed;

use Acme\Tickets\Factory\TicketFactory;
use Acme\Tickets\Factory\TicketReplyFactory;
use Hampel\Seeder\Factory\UserFactory;
use Hampel\Seeder\Seed\AbstractSeeder;

class TicketSeeder extends AbstractSeeder
{
    public function run()
    {
        $customers = UserFactory::new()->count($this->count('customers', 20))->create();
        $agents = UserFactory::new()
            ->state(['is_moderator' => true, 'is_staff' => true])
            ->count($this->count('agents', 3))
            ->create();
        $agentIds = array_values($agents->pluckNamed('user_id'));

        TicketFactory::new()
            ->recycle($customers)
            // one ticket in five is closed, one in ten is urgent
            ->sequence(
                [], [], [], [],
                ['status' => 'closed'],
                [], [], [], [],
                ['priority' => 'urgent']
            )
            // each ticket is assigned to one of the agents
            ->state(function () use ($agentIds)
            {
                return ['assigned_user_id' => $agentIds[array_rand($agentIds)]];
            })
            // and gets between none and eight replies, from the same customers
            ->has(TicketReplyFactory::new()->count([0, 8]))
            ->count($this->count('tickets', 200))
            ->create();
    }
}

Bash:

php cmd.php seeder:run Acme/Tickets:Ticket -p tickets=2000 -p customers=150 --span=1y --seed=42

That gives a year of tickets to page through, sort, search and report on, and --seed makes the names and text the same on every run, so a screenshot or a failing case can be reproduced. A parameter the seeder never asks for, such as a mistyped -p ticket=2000, fails the run and is named.

One case, built by hand

Factories are as useful for a single awkward case as for a thousand rows. To see how a ticket page handles a banned customer with an urgent ticket and a long history:

PHP:

$customer = UserFactory::new()->banned()->create();

TicketFactory::new()
    ->for($customer)
    ->urgent()
    ->has(TicketReplyFactory::new()->count(60))
    ->create(['subject' => 'Sixty replies and a banned customer']);

Three things worth knowing

Only what a factory's create() makes is recorded. A row you save yourself inside a seeder is not, and seeder:clean will not remove it. Give it a factory, or make sure your add-on deletes it when its owner is deleted.
Cleaning calls your entity's own delete. If deleting a ticket in your add-on removes its replies and updates its counters, cleaning does too. Where it leaves something behind, override the factory's remove(). seeder:validate warns when a table of yours does not return to its count. It counts rows, not files: if your content writes files, delete them in remove() and count them yourself before seeding and after cleaning.
Leave Seed and Factory out of your release zip. They are development tooling, like tests, and they need Seeder installed. Remove them in your build.json.

For tables whose rows stand alone and number in the tens of thousands, such as a log, there is a bulk path that writes many rows per query. For code that reads XenForo's request - the address or headers a registration arrived on - Context::withRequest() runs it on a request made from the values you give. The README covers both, with the full list of factory methods.

Before seeding a copy of a live forum

Turn mail off first. Seeder sends nothing itself, but a forum that can send mail will later mail its real members about seeded content, such as a digest or a watched thread. seeder:validate warns when mail is on.

PHP:

$config['enableMail'] = false;

Limits

Seeded content shows as active today in trending content, whatever date it carries.
Daily statistics are not touched. If you rebuild them while seeded content exists, rebuild them again after cleaning.
Seeding and cleaning each take about a second per fifty posts.
Another add-on's own tables are that add-on's to clean. One that does not clean up when a member or post is deleted shows as a warning in seeder:validate.
An HTML widget has a template of its own. Saving or deleting one marks every style as changed and empties the cached CSS, as editing a template in the admin control panel does. The Widget seeder makes none unless asked with html.
Deleting a user group does the same, so cleaning a batch with user groups in it - members given permissions, or permission groups - empties the cached CSS too.
A seeded member's preferences are the forum's default registration values, where XenForo ships receive admin email off.
The option that runs other add-ons' hooks on user upgrades (hooks=1) has not been tried on a real forum.
Registering through XenForo's service (registered=1) runs whatever other add-ons do when a member registers, which can include sending the member's details out of the forum. XenForo's own mail, welcome message and trophies are off for it.

Licence

MIT.

Related resources

Reactions: FlowRider, lazy llama, nocte and 5 others

Beta Seeder: test data generator 1.0.0 beta 10

This is a beta

Requirements

What it creates

What it will not do

Installation

Usage

Your own seeders and factories

Use the built-in factories to set up a case you need to test

A factory for your own content

A seeder that builds a believable set

One case, built by hand

Three things worth knowing

Before seeding a copy of a live forum

Limits

Licence

Ratings

More resources from Sim

Share this resource

Latest updates

v1.0.0 beta 10

v1.0.0 beta 9

v1.0.0 beta 8

We value your privacy

Beta Seeder: test data generator 1.0.0 beta 10

This is a beta​

Requirements​

What it creates​

What it will not do​

Installation​

Usage​

Your own seeders and factories​

Use the built-in factories to set up a case you need to test​

A factory for your own content​

A seeder that builds a believable set​

One case, built by hand​

Three things worth knowing​

Before seeding a copy of a live forum​

Limits​

Licence​

Ratings

More resources from Sim

Share this resource

Latest updates

v1.0.0 beta 10

v1.0.0 beta 9

v1.0.0 beta 8

We value your privacy

This is a beta

Requirements

What it creates

What it will not do

Installation

Usage

Your own seeders and factories

Use the built-in factories to set up a case you need to test

A factory for your own content

A seeder that builds a believable set

One case, built by hand

Three things worth knowing

Before seeding a copy of a live forum

Limits

Licence