Well, as
@Max Taxable has already pointed out - it is "safe" to delete the content, nothing too bad/irreversible will happen.
However, your XenForo installation will stop working as it has no template code to render unless you recompile templates.
This is not much different from other software/frameworks that also compile templates to executable PHP code, the only difference is that other software might automatically recompile if compiled ("cached") templates are missing - XenForo does not do that.