-40 armor will cause you to take double damage from sources that do not ignore armor.
Because of the heavy penalty, it would be wise to combine Healing Signet with a defensive skill such as Disciplined Stance, otherwise the extra damage may outweigh the healing.