Templates must be rebuilt after uninstalling an add-on, there is no way around that.
It is possible though to do a single rebuild.
If you are troubleshooting to identify which add-on is problematic, why not just disable them rather than uninstall them?
Once you have identified the add-on, it can be uninstalled.