The main difference is there are several separate template systems. There's obviously a separation for email templates too.
Functionally, there are differences in the syntax that each support. You can use Admin form helpers in admin templates, e.g. <xen:form>, <xen:textboxunit> etc. whereas you can't use those in public templates.
In addition to that, the public templates have much more functionality. They have styling support, edit history, merging support etc.