Currently, I would need to extend \XF\Admin\Controller\Asset::getAssetPermissionMap() in order to do admin permission check instead of a simple code event listener.
Something like user_delete_clean_init which passes $deletes by reference should do the trick.