XF 2.0 Composer autoload method

[Jake B.]

This is what I'm using currently to autoload any classes I'm requiring through Composer for add-ons. It hasn't been thoroughly tested yet to make sure everything works properly if two add-ons are including the same packages, but from my brief testing it seems to function. For the most part it's pulled directly out of Composer so I can't imagine there being any major issues

Just create an app_setup event listener that points to this method:

public static function autoloadClasses()
{
    $composerDirectory = \XF::getAddOnDirectory() . '/VendorPrefix/AddOn/vendor/composer';
    $loader = \XF::$autoLoader;
    $map = require $composerDirectory . '/autoload_namespaces.php';
    foreach ($map as $namespace => $path) {
        $loader->set($namespace, $path);
    }

    $map = require $composerDirectory . '/autoload_psr4.php';
    foreach ($map as $namespace => $path) {
        $loader->setPsr4($namespace, $path);
    }

    $classMap = require $composerDirectory . '/autoload_classmap.php';
    if ($classMap) {
        $loader->addClassMap($classMap);
    }
}

obviously you'll replace VendorPrefix/AddOn with your Add-on's ID, you can probably also use __DIR__ instead if the file is in the root directory of your add-on, or __DIR__ . '/../' if it's in an Listeners directory. This will load namespaces, classmaps, etc directly from vendor/composer/* and register all of the classes for you automagically

If anyone more knowledgeable with the inner workings of XF's autoloader and composer sees any issues with this do let me know

 

[Sim]

The way I structure my addons is to have a composer.json in the root of the addon and I run composer update from the root directory vendor folder containing the composer autoload files.

I then add the following Composer.php class to the root of my addon.

<?php

namespace MyAddon;

class Composer
{
    public static function autoloadNamespaces(\XF\App $app, $prepend = false)
    {
        $namespaces = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_namespaces.php';

        if (!file_exists($namespaces))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $namespaces);
        }
        else
        {
            $map = require $namespaces;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->add($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadPsr4(\XF\App $app, $prepend = false)
    {
        $psr4 = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_psr4.php';

        if (!file_exists($psr4))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $psr4);
        }
        else
        {
            $map = require $psr4;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->addPsr4($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadClassmap(\XF\App $app)
    {
        $classmap = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_classmap.php';

        if (!file_exists($classmap))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $classmap);
        }
        else
        {
            $map = require $classmap;

            if ($map)
            {
                \XF::$autoLoader->addClassMap($map);
            }
        }
    }

    public static function autoloadFiles(\XF\App $app)
    {
        $files = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_files.php';

        if (!file_exists($files))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $files);
        }
        else
        {
            $includeFiles = require $files;

            foreach ($includeFiles as $fileIdentifier => $file)
            {
                if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
                    require $file;

                    $GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
                }
            }
        }
    }
}

Finally, I add a code event listener for the app_setup event which executes the following code to actually perform the autoload:

    public static function appSetup(App $app)
    {
        Composer::autoloadNamespaces($app);
        Composer::autoloadPsr4($app);
        Composer::autoloadClassmap($app);
        Composer::autoloadFiles($app);
    }

It's important to run all four autoloaders because different packages will utilise different autoload mechanisms. This is basically what the composer autoloader does itself.

You can see this example in operation in my Guzzle6 addon (also:
https://xenforo.com/community/resources/guzzle6-for-xenforo-2.5578/)

 

[Jake B.]

Ah looks (for the most part) to be almost exactly what I did yours just seems more organized, and I apparently forgot to copy over the autoload_files.php part. How does the XF AutoLoader handle if two add-ons are using the same
package (or two separate versions of the same package)?

 

[Jake B.]

Ended up making a bit of a change to what you have to be more generic, have added it to one of our core files that gets included with all add-ons:

<?php

namespace ThemeHouse\Core;

class AutoLoad
{
    public static function autoloadComposerPackages($addOnId)
    {
        $composerDir = \XF::getAddOnDirectory() . '/' . $addOnId .'/vendor/composer';
        $app = \XF::app();

        self::autoloadNamespaces($composerDir, $app);
        self::autoloadPsr4($composerDir, $app);
        self::autoloadClassmap($composerDir, $app);
        self::autoloadFiles($composerDir, $app);
    }

    public static function autoloadNamespaces($composerDir, \XF\App $app, $prepend = false)
    {
        $namespaces = $composerDir . DIRECTORY_SEPARATOR . 'autoload_namespaces.php';

        if (!file_exists($namespaces)) {
            $app->error()->logError('Missing vendor autoload files at %s', $namespaces);
        } else {
            $map = require $namespaces;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->add($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadPsr4($composerDir, \XF\App $app, $prepend = false)
    {
        $psr4 = $composerDir . DIRECTORY_SEPARATOR . 'autoload_psr4.php';

        if (!file_exists($psr4)) {
            $app->error()->logError('Missing vendor autoload files at %s', $psr4);
        } else {
            $map = require $psr4;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->addPsr4($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadClassmap($composerDir, \XF\App $app)
    {
        $classmap = $composerDir . DIRECTORY_SEPARATOR . 'autoload_classmap.php';

        if (!file_exists($classmap)) {
            $app->error()->logError('Missing vendor autoload files at %s', $classmap);
        } else {
            $map = require $classmap;

            if ($map) {
                \XF::$autoLoader->addClassMap($map);
            }
        }
    }

    public static function autoloadFiles($composerDir, \XF\App $app)
    {
        $files = $composerDir . DIRECTORY_SEPARATOR . 'autoload_files.php';

        if (!file_exists($files)) {
            $app->error()->logError('Missing vendor autoload files at %s', $files);
        } else {
            $includeFiles = require $files;

            foreach ($includeFiles as $fileIdentifier => $file) {
                if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
                    require $file;

                    $GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
                }
            }
        }
    }

}

and now in the app_setup listener it can just call \ThemeHouse\Core\AutoLoad::autoloadComposerPackages('ThemeHouse/AddOnId')

 

[Sim]

How does the XF AutoLoader handle if two add-ons are using the same
package (or two separate versions of the same package)?

The magic is in the $prepend parameter to the add and addPsr4 methods.

    /**
     * Registers a set of PSR-0 directories for a given prefix, either
     * appending or prepending to the ones previously set for this prefix.
     *
     * @param string       $prefix  The prefix
     * @param array|string $paths   The PSR-0 root directories
     * @param bool         $prepend Whether to prepend the directories
     */
    public function add($prefix, $paths, $prepend = false)
    {
        ...

By default, the autoloader appends the directories to any existing ones already found, so core vendor files take priority over addon vendor files.

So the XF autoloader (which by default is the Composer Autoloader from /src/vendor/composer/ClassLoader.php) will load its own autoload files first from /src/vendor/composer/autoload_namespaces.php and then /src/vendor/composer/autoload_psr4.php.

We then call the Composer autoloader again via our code event listener to load our addon's vendors autoload files - but it depends on how we set the $prepend variable as to what it does next.

If we leave $prepend set to the default value of false, it appends our new directories to any existing directories, which means that any classes autoloaded from the core vendor directory will take precedence. Similarly, any addons where their code event listener fires before ours and they utilise the class loader - will take priority over ours because ours will be added to the end of the list. This is where execution priority may become important.

Alternatively, if we set $prepend to true, our classes will be loaded first and will take priority.

So for my Guzzle6 library, I actually use the following app_setup code event listener code, because I need my vendor classes to take priority over the vendor classes loaded by the core:

    public static function appSetup(App $app)
    {
        Composer::autoloadNamespaces($app, true); // prepend our vendor classes so they load first!!
        Composer::autoloadPsr4($app, true); // prepend our vendor classes so they load first!!
        Composer::autoloadClassmap($app);
        Composer::autoloadFiles($app);
    }

The key with the autoloader code is that it first builds a list of possible directory locations for the classes it has been asked to load, and then goes through them one by one to try and find a match. Before it loads a class, if first checks whether it has already been loaded, and if so, doesn't try to load it again.

So if you need to over-ride an existing vendor class with one from your addon, you want to be the first directory it finds with your class in it - hence being prepended before the beginning of the list rather than appended to the end of the list.

 

[LandNetwork]

The way I structure my addons is to have a composer.json in the root of the addon and I run composer update from the root directory vendor folder containing the composer autoload files.

I then add the following Composer.php class to the root of my addon.

<?php

namespace MyAddon;

class Composer
{
    public static function autoloadNamespaces(\XF\App $app, $prepend = false)
    {
        $namespaces = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_namespaces.php';

        if (!file_exists($namespaces))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $namespaces);
        }
        else
        {
            $map = require $namespaces;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->add($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadPsr4(\XF\App $app, $prepend = false)
    {
        $psr4 = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_psr4.php';

        if (!file_exists($psr4))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $psr4);
        }
        else
        {
            $map = require $psr4;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->addPsr4($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadClassmap(\XF\App $app)
    {
        $classmap = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_classmap.php';

        if (!file_exists($classmap))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $classmap);
        }
        else
        {
            $map = require $classmap;

            if ($map)
            {
                \XF::$autoLoader->addClassMap($map);
            }
        }
    }

    public static function autoloadFiles(\XF\App $app)
    {
        $files = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_files.php';

        if (!file_exists($files))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $files);
        }
        else
        {
            $includeFiles = require $files;

            foreach ($includeFiles as $fileIdentifier => $file)
            {
                if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
                    require $file;

                    $GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
                }
            }
        }
    }
}

Finally, I add a code event listener for the app_setup event which executes the following code to actually perform the autoload:

    public static function appSetup(App $app)
    {
        Composer::autoloadNamespaces($app);
        Composer::autoloadPsr4($app);
        Composer::autoloadClassmap($app);
        Composer::autoloadFiles($app);
    }

It's important to run all four autoloaders because different packages will utilise different autoload mechanisms. This is basically what the composer autoloader does itself.

You can see this example in operation in my Guzzle6 addon (also:
https://xenforo.com/community/resources/guzzle6-for-xenforo-2.5578/)

Not meaning to necropost, but this is a brilliant solution which I will use if you don't mind. Thank you very much.

 

[BoostN]

How does the autoload_files.php get generated?

 

[Sim]

How does the autoload_files.php get generated?

It gets automatically generated from the contents of the composer.json file after running a composer install or a composer update command.

The contents of the autoload_files.php comes from the "files" autoload mechanism - refer to https://getcomposer.org/doc/04-schema.md#files

For example, the following JSON snippet is from the Guzzle 6 composer.json file where it includes the src/functions_include.php file which defines global functions which aren't contained within a class.

    "autoload": {
        "files": ["src/functions_include.php"],
        "psr-4": {
            "GuzzleHttp\\": "src/"
        }
    },

... this generates array entries in the autoload_files.php file which are "required" by the Composer autoloader at runtime (or in my code above, you can see where it requires the file in the autoloadFiles function).

Here is the autoload_files.php file from my Guzzle 6 addon vendor folder:

<?php

// autoload_files.php @generated by Composer

$vendorDir = dirname(dirname(__FILE__));
$baseDir = dirname($vendorDir);

return array(
    'c964ee0ededf28c96ebd9db5099ef910' => $vendorDir . '/guzzlehttp/promises/src/functions_include.php',
    'a0edc8309cc5e1d60e3047b5df6b7052' => $vendorDir . '/guzzlehttp/psr7/src/functions_include.php',
    '37a3dc5111fe8f707ab4c132ef1dbc62' => $vendorDir . '/guzzlehttp/guzzle/src/functions_include.php',
);

... there are two other entries in there from other libraries as well.

 

[BoostN]

Thanks @Sim . I'm new to Composer, so that brings me to another question.

The package I'm trying to reference is (Monolog), it doesn't have the same functions_include.php file. I understand it's probably not the same for every package, but I'm having problems figuring out which file I should reference.

"autoload": {
        "files": ["monolog/src/???.php"]
    }

I removed the package from the addon, the added this composer file. Then did a "composer update". Everything installs fine, but I'm still missing the autoload_files.php.

{
    "name": "AutoLoad",
    "description": "AutoLoad",
    "license": "MIT",
    "authors": [
        {
            "name": "Example",
            "email": "test@test.com"
        }
    ],
    "autoload": {
        "psr-4": {"Monolog\\": "src/Monolog"}
    },
    "require": {
        "monolog/monolog": "1.*"
    }
}

 

[Sim]

@BoostN - is this for XF2 ?

I do have a Monolog addon which I can show you for example for XF1 - but since XF1 didn't use composer internally, it was actually easier, because all I had to do was include the vendor/autoload.php file in my code, which does all the Composer autoloading magic for you.

You actually only need the "require" bit of the composer.json file:

    "require": {
        "monolog/monolog": "1.*"
    }

... and then you run composer update in the directory where the composer.json file is, to generate the vendor directory.

Then in my XF1 init_dependencies code event I simply require the vendor/autoload.php file.

For XF2, because we already have autoloading as part of the core, we need to supplement that with our own autoload - hence my code examples earlier in this thread.

You start the same way with a simple composer.json file that just has a "require" (none of the other stuff I have in there is actually required), run composer update to generate the vendor directory, and then in the app_setup code event listener, you simply call the Composer autoload code to include the vendor files

Example from Guzzle6\Listener::appSetup which is executed in the app_setup code event:

<?php

namespace Guzzle6;

use XF\App;
use XF\Container;

class Listener
{
    public static function appSetup(App $app)
    {
        Composer::autoloadNamespaces($app, true);
        Composer::autoloadPsr4($app, true);
        Composer::autoloadClassmap($app);
        Composer::autoloadFiles($app);
    }

... where the Guzzle6\Composer.php file is exactly what I posted in post #2 above (with name space changed to suit addon).

So when I get around to porting my Monolog addon for XF2, it will operate exactly the same way as my Guzzle6 code above does (different namespace for a different addon of course), but the remaining code will be identical, I will literally just copy and paste the Composer.php class and change the namespace and use the same appSetup code snippet from above to call it from the app_setup code event.

If there are multiple libraries required for the addon - there is no extra work, you simply list them all in the composer.json file, run composer update and they automagically get included by the code event autoloading routine.

 

[BoostN]

Yes, for XF2:

I ran it again, and I'm getting this ACP Error:

ErrorException: Missing vendor autoload files at %s src\XF\Error.php:75
Generated by: Unknown account Jan 9, 2018 at 4:57 AM
Stack trace
#0 src\addons\BoostN\AutoLog\Composer.php(68): XF\Error->logError('4:Missing vendo...', 'C:\\xampp\\htdocs...')
#1 src\addons\BoostN\AutoLog\Listener.php(15): BoostN\AutoLog\Composer::autoloadFiles(Object(XF\Admin\App))
#2 [internal function]: BoostN\AutoLog\Listener::appSetup(Object(XF\Admin\App))
#3 src\XF\Extension.php(67): call_user_func_array(Array, Array)
#4 src\XF\App.php(2341): XF\Extension->fire('app_setup', Array, NULL)
#5 src\XF\App.php(1493): XF\App->fire('app_setup', Array)
#6 src\XF\Admin\App.php(40): XF\App->setup(Array)
#7 src\XF.php(312): XF\Admin\App->setup(Array)
#8 src\XF.php(324): XF::setupApp('XF\\Admin\\App')
#9 admin.php(13): XF::runApp('XF\\Admin\\App')
#10 {main}

It's coming from line 62 in my file, like yours:
https://bitbucket.org/hampel/guzzle...&fileviewer=file-view-default#Composer.php-62

 

[Sim]

Yes, for XF2:

I ran it again, and I'm getting this ACP Error:

ErrorException: Missing vendor autoload files at %s src\XF\Error.php:75
Generated by: Unknown account Jan 9, 2018 at 4:57 AM
Stack trace
#0 src\addons\BoostN\AutoLog\Composer.php(68): XF\Error->logError('4:Missing vendo...', 'C:\\xampp\\htdocs...')
#1 src\addons\BoostN\AutoLog\Listener.php(15): BoostN\AutoLog\Composer::autoloadFiles(Object(XF\Admin\App))
#2 [internal function]: BoostN\AutoLog\Listener::appSetup(Object(XF\Admin\App))
#3 src\XF\Extension.php(67): call_user_func_array(Array, Array)
#4 src\XF\App.php(2341): XF\Extension->fire('app_setup', Array, NULL)
#5 src\XF\App.php(1493): XF\App->fire('app_setup', Array)
#6 src\XF\Admin\App.php(40): XF\App->setup(Array)
#7 src\XF.php(312): XF\Admin\App->setup(Array)
#8 src\XF.php(324): XF::setupApp('XF\\Admin\\App')
#9 admin.php(13): XF::runApp('XF\\Admin\\App')
#10 {main}

It's coming from line 62 in my file, like yours:
https://bitbucket.org/hampel/guzzle...&fileviewer=file-view-default#Composer.php-62

You won't need to use the "files" autoload directive in your composer.json

Just try using literally the following as your composer.json file:

"require": {
    "monolog/monolog": "1.*"
}

... and then run composer update again.

 

[Sim]

Hmm - turns out that the vendor/composer/autoload_files.php must only be generated when there is actually a "files" autoload directive somewhere in the vendor requirements.

I've changed Composer::autoloadFiles to not fail when this file does not exist.

<?php

namespace MyAddon;

class Composer
{
    public static function autoloadNamespaces(\XF\App $app, $prepend = false)
    {
        $namespaces = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_namespaces.php';

        if (!file_exists($namespaces))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $namespaces);
        }
        else
        {
            $map = require $namespaces;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->add($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadPsr4(\XF\App $app, $prepend = false)
    {
        $psr4 = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_psr4.php';

        if (!file_exists($psr4))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $psr4);
        }
        else
        {
            $map = require $psr4;

            foreach ($map as $namespace => $path) {
                \XF::$autoLoader->addPsr4($namespace, $path, $prepend);
            }
        }
    }

    public static function autoloadClassmap(\XF\App $app)
    {
        $classmap = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_classmap.php';

        if (!file_exists($classmap))
        {
            $app->error()->logError('Missing vendor autoload files at %s', $classmap);
        }
        else
        {
            $map = require $classmap;

            if ($map)
            {
                \XF::$autoLoader->addClassMap($map);
            }
        }
    }

    public static function autoloadFiles(\XF\App $app)
    {
        $files = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_files.php';

        if (!file_exists($files))
        {
            $includeFiles = require $files;

            foreach ($includeFiles as $fileIdentifier => $file)
            {
                if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
                    require $file;

                    $GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
                }
            }
        }
    }
}

 

[Sim]

You know that thing when you spend all afternoon trying to debug some code that should just work ... but doesn't?

... and you're banging your head against the wall and getting increasingly frustrated and nothing makes sense at all?

... and then sometime after 10pm that night you finally notice you accidentally left a ! in an if statement you changed the other day, such that it was never going to work.

*sigh*

The autoloadFiles function above should read:

    public static function autoloadFiles(\XF\App $app)
    {
        $files = __DIR__ . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'composer' . DIRECTORY_SEPARATOR . 'autoload_files.php';

        if (file_exists($files)) // <-- oops!!
        {
            $includeFiles = require $files;

            foreach ($includeFiles as $fileIdentifier => $file)
            {
                if (empty($GLOBALS['__composer_autoload_files'][$fileIdentifier])) {
                    require $file;

                    $GLOBALS['__composer_autoload_files'][$fileIdentifier] = true;
                }
            }
        }
    }

Apologies if this caused any problems for anyone - it's driven me mad today.

 

[Sim]

Just FYI - I have written a full tutorial on how to include Composer packages in your XenForo addons, which also includes some discussion about the gotchas and caveats in doing so.

Using Composer Packages in XenForo 2 Addons Tutorial

In that tutorial, I also mention the licensing of the code I've written (which is also published in this thread) - which I am providing under an MIT license. This gives you full rights to use and/or modify the code free of charge, including for commercial purposes - provided that the license and copyright information is included. I only mention this because someone asked about using my code - so I thought I would make it explicit, you are allowed to!