Caches when running multiple frontend servers

Christer L

I am runing XF 2.1 on four frontend servers behind a load balancer (using a storage adapter so attachments etc are shared)
When I change templates or phrases, these changes are not visible on all servers - so I need to rebuild caches on each

I have been using xf:rebuild-master-data, which seems to do the trick - but it is quite slow in the rebuilding permissions phase

Is it necessary to also rebuild permissions on each host, if I have not changed permissions?
Is there a faster way to just rebuild caches for phrases and templates?

Or, is there a better way to solve this - like a shared cache, what is available?