Did a bit of searching for this and came up short. After some testing I've figured out a way to do this and thought I'd share.
@Brogan if this is of interest please move to Tip and Guides.
VERSION: Tested with v2.1.9 without using other add-ons.
PROBLEM: You want to manually assign/remove...