XF 1.2 Template Modifications and Comparison

Anyone who has developed add-ons for XenForo 1.1 knows that template hooks are a pain. They're a pain for us too, to say the least. Plenty of people have installed the Template Modification System add-on. We've been planning a TMS-like system for some time, before the XenForo add-on was ever actually released.

XenForo 1.2 will deprecate the template hook system and introduce it's own template modification system to make it easier for add-on developers to change templates. Note that I'm primarily focused on add-on developers here. It is possible to create your own custom template modifications if you're not an add-on developer, but unless you have multiple styles, I'd recommend just making the changes to the templates directly.

Let's look at what a template modification is:

ss-2013-04-12_11-35-58.png


It's simply a "find this" and "replace with that" style replacement. You can do this to any template. If your find matches multiple times in a single template, it will replace each occurrence.

Applying a template modification does not actually modify the template, so it will not show up as customized and it will be automatically reapplied (if possible) when upgrading.

For those of you that want technical details, the modifications are applied before a template is parsed. If the modifications cause a template compilation error, they are automatically disabled.

Here's what the template modification overview looks like:

ss-2013-04-12_11-41-07.png


All of the template modifications will be listed here, grouped by add-on and ordered by template. The checkbox enables/disables the modification. Note that if the modification is part of an add-on (and you're not in debug mode), you will not be able to edit the modification here but you can enable/disable it. More about this when editing a template.

The 3 numbers indicate the number of times the modification matched (green), the number of times the modification failed to match (grey) and the number of times the modification caused an error (red).

Editing a Template
There are a few new bits when editing a template as well.

ss-2013-04-12_11-43-10.png


There are 3 new things here:
  • The "prevent" checkbox. If selected, modifications will be never be applied to this template. This can be used to quickly check a template without modifications or to adjust the modifications manually.
  • The "view template modifications" link. I'll discuss this below.
  • The "view custom changes" link. This will simply display a comparison of the parent version of the template (probably the master version) and the current version so you can identify the differences.
When you select view template modifications, you'll see an overlay like this:
ss-2013-04-12_11-43-25.png

This shows you a comparison of how the template modifications will be applied to this template.
If you choose to manually apply modifications, the template modifications will be "materialized" into the template - they will actually be written into the template and automatic application of modifications will be disabled in this template. You can then manually edit the modifications if you wish. (Obviously if you upgrade an add-on and they've changed their modifications or you install a new add-on that changes this template, you won't be getting those new modifications so this should be used with care.)

Admin and Email Templates
These aren't left out. Add-on developers will have access to very similar systems that allow the admin and email templates to be manipulated in the same way.
We look forward to more dynamic add-ons with XenForo 1.2. :)
 
Last edited by a moderator:

Erik

Well-known member
+1 to getting this working on an at least per-style basis. My hope is that their silence on the issue means that something more might be coming for styles, perhaps integration with the history system and proper version-tracking and 3-way merging.

Using regular expressions to make modifications to HTML or CSS is nice but is bound to break at some point. In the case of styles, tracking template versions (including the master templates) and doing a proper 3-way merge when updating is much more semantic and less likely to fail over time. Developers/admins might occasionally have to resolve conflicts, but at least it is clear what is going on rather than working with "dumb" regular expressions that might cause unintended changes behind your back (or simply fail without you noticing).

Disregard, I'm retarded. This already exists! http://xenforo.com/community/threads/template-merging-and-history.48846/page-2#post-523807
 
Last edited:

Chris D

XenForo developer
Staff member
It's not possible to cause unintended changes or have them fail without noticing.

That's why there's a test button and indications as to how many times it applies to a template.
 

Erik

Well-known member
It's not possible to cause unintended changes or have them fail without noticing.

That's why there's a test button and indications as to how many times it applies to a template.

So if you update XenForo or a 3rd-party style, are you supposed to keep a record of how many times a TM has been applied and then verify those numbers on each update to make sure it hasn't been applied somewhere accidentally?

Regardless, a 3-way merge is much more semantic (and therefore accurate) because it has knowledge of the source tree and the whole file. If something can't be merged without a conflict (the equivalent to a well-written, best-case regular expression failing to apply because of changes), you can resolve the conflict intelligently right there with the full source of all relevant files at your disposal.

The problem with using regular expressions to make modifications like this is that the developer or user has to decide how much context to use (in other words, how much do you try to match in your regular expression?).

Use too much context (an exaggerated example would be matching the entire template), and you risk unrelated changes breaking your template modification because the regex no longer matches. Use too little context, and you risk the modification being applied in areas you don't want it to.

Doing a 3-way merge, on the other hand, is much more intelligent! Because it has access to the full source of the ancestor and both child derivatives (I'm talking about a tree as in version history, not the XenForo template system), it can figure out exactly what the differences are and apply changes only where they need to be applied, while leaving other things alone and minimizing conflicts.

The only advantage a regular-expression-based template modification system has over version history/control and a 3-way merge is that changes can be applied across multiple templates simultaneously with only one modification. However, I can't see a common use case for this when developing styles - most changes are very specific and large numbers of changes that need to be repeated across templates would raise concerns that things aren't very DRY.

I can see how this could be very useful for addon developers, although in my mind the jury's still out on whether this is actually a better solution than well-thought-out template hooks (for the reasons John mentioned earlier in the thread: style developers can guarantee that all the template hooks will always be there in the place they need to be, but depending on how addon developers write their regex's, it could be very easy to break an addon's template modifications).

However, for style developers, template modifications are a half-assed solution to what we all really want: proper version control and 3-way merging. A 3-way merge is an all-around technically better solution than template modifications. If anyone still doesn't understand why, I would suggest reading up on version control and difference merging - it's very cool stuff.

- Erik
 
Last edited:

Rob

Well-known member
It's unrealistic to expect XenForo to create an importer for a third party add-on which only a small percentage of license holders use.
It's time which could be better spent elsewhere.

Actually Brogan, I very strongly disagree with this.

Through the creation of the marketplace / resources area it is xenforo itself that led to the growth of the add on community. TMS is actually the 5th most downloaded add on and has been installed by well over a thousand license holders. Many add ons support TMS.

Given the relative success of TMS I think that if the responsibility lies with anyone at all, then it absolutely is the responsibility of xenForo and not that of the add on developer. It is xenforo that has decided to adopt this as native functionality and in doing so has created potential upgrade conflicts for from what I can see at least 1000 users. It is surely not the responsibility of the add on creator to create another add on which will help users remove his add on - that is thoroughly ludicrous.

Should the add on developer have the forsight to look into a crystal ball, see that his add on will become hugely popular in this case and then one day adopted natively leaving all his add on users practically up sh!t creek then add ons wouldn't get made.

Xenforo has access to the TMS code, they no doubt inspected it at some depth during the making of this feature and the very least they can do is to provide an upgrade path.

For the record, I do not run TMS but if I did I would be quite peeved.

In xenforo's defence, I have heard no official line on this at all so it may even be in the bag for another time.
 

Chris D

XenForo developer
Staff member
Your entire post was a waste of time, unfotunately, @Rob.

XenForo won't be developing a TMS converter because @guiltar himself has created a converter for the existing TMS add-on. It would be pointless for XenForo now to create one.

I think it's even already available from @guiltar's github repo.
 

Jeremy

in memoriam 1991-2020
@Rob, its rather a moot point since both TMS and XenForo changed their syntaxes to allow both to coexist. This could also be completely avoided in the future if add-ons authors prefix their add-ons. This is even advocated by XenForo and its developers, since it avoids situations such as this.

Mike has stated that he did not look at TMS code when developing, so no, they didn't realize that the TMS didn't prefix their code and that they both chose a logical name for it. I have seen several add-on developers already advocate that they would create an importer for our add-on to XenForo's official. For how quickly add-ons can be made and released, I agree with Paul, it seems like a waste of valuable time that could be used elsewhere.
 

Rob

Well-known member
I'll agree its a moot point (having picked up on that after i posted) but still, its an attitude thing I was poking holes at not xenforo or guiltar.
 

Jaxel

Well-known member
Will this completely eliminate the template hook system? Or will it be in addition to it?

Which method would be more resource efficient?
 

Jeremy

in memoriam 1991-2020
Jaxel:

XenForo 1.2 will deprecate the template hook system and introduce it's own template modification system to make it easier for add-on developers to change templates. Note that I'm primarily focused on add-on developers here. It is possible to create your own custom template modifications if you're not an add-on developer, but unless you have multiple styles, I'd recommend just making the changes to the templates directly.

Its replacing the template hooks and no knew hooks will be added going forward; but they will remain function for the time being. You should transfer them over to be template modifications. If I remember correctly, they are applied on install and as templates are updated, not run time as with template hooks, so they should theoretically work faster. I haven't look into the internals, so don't quote me, but I would say the new modifications are better.
 

Jaxel

Well-known member
Okay, so I'm looking at the replacement system and I see this:
  • PHP Callback
    The search text must be a full regular expression. The replace field must be in the form of className::methodName to call. The callback will receive one argument, an array of matches from the regular expression. It must return the updated template string.
Is it possible to get a slight change to this? Instead of only one argument, an array of matches, can we also get a second argument, the name of the template being searched?

Also, this new system means there are no hookParams, like thread info... now what do we do?
 
Last edited:

Jeremy

in memoriam 1991-2020
This new system occurs before compilation, if an edit needs to occur in only certain instances, you will need to use conditionals. Also, the template name should be inconsequential, are you using the same callback for more than one edit?
 

DRE

Well-known member
wtf is this modification key for?



"This is used to uniquely identify a modification across add-on upgrades."
 

Lisa

Well-known member
Just found out you can't apply TMs for specific style(s). That's disappointing.
If you tick the checkbox within the template of the style you don't want a modification to work in.... you technically can :)

templatechecked.png


If you go to http://xenadmins.com and compare the online users sidebar module on the custom style to the default xenforo style, you'll see the TM edit works in the custom style but is switched off for the default.
 
Top