It doesn't "add" it as such. It's already there. Presumably you're changing a INT column which already has UNSIGNED to a VARCHAR.
For the most part, when you're changing a column, it attempts to keep as much of the original column config as possible. In addition to setting unsigned to false, the other workaround is to call ->resetDefinition() on it.
That said, I agree it would be illogical for a varchar column to keep the unsigned property so we'll try and address that. But, just be aware that sometimes when changing a column, you'll need to reset its definition or manually adjust the existing properties of it.