I think it should stay, so people know they are missing something. Actually, I think if you go to the Alerts page, I'd display a message "You have all forms of alerts disabled. You will not be notified of any new messages, forum posts, or other content types that may interest you."