My main problem at the moment is, that the color picker is never cloned to begin with, although you're probably right, that even when it would be cloned, it would probably not be initialized. Anyhow, as said, it's not even cloned to begin with.
As far as I can see, the FieldAdder is listening for the change event on the color picker input element. I can't tell why, but apparently the color picker itself doesn't trigger this event when changing the value of the input field and therefor bypasses the field adder?