I'm fairly sure you can't limit this to a single template. The template system plugs into the hook setting that you have configured in your language format.
With that said, I'm sure someone could make an add-on to plug into that hook.
There's a list of usable format variables I found in XenForo_Locale::getFormattedDateInternal. Not sure if they're supposed to work with xen:datetime too, cause I had a lot of trouble finding a combination that did work. But glad to see it was possible.