The permissions analyser will identify how the final value is being calculated.
If the member is in another user group with the permission set to Yes, that will override the No.
In this case you would have to set it to Never for that member, to override the Yes.