יובל אלוביץ

Despite their accuracy, neural network-based classifiers are still prone to manipulation through adversarial perturbations. These perturbations are designed to be misclassified by the neural network while being perceptually identical to some valid inputs. The vast majority of such attack methods rely on white-box conditions, where the attacker has full knowledge of the attacked network's parameters. This allows the attacker to calculate the network's loss gradient with respect to some valid inputs and use this gradient in order to create an adversarial example. The task of blocking white-box attacks has proved difficult to address. While many defense methods have been suggested, they have had limited success. In this article, we examine this difficulty and try to understand it. We systematically explore the capabilities and limitations of defensive distillation, one of the most promising defense mechanisms against adversarial perturbations suggested so far, in order to understand this defense challenge. We show that contrary to commonly held belief, the ability to bypass defensive distillation is not dependent on an attack's level of sophistication. In fact, simple approaches, such as the targeted gradient sign method, are capable of effectively bypassing defensive distillation. We prove that defensive distillation is highly effective against nontargeted attacks but is unsuitable for targeted attacks. This discovery led to our realization that targeted attacks leverage the same input gradient that allows a network to be trained. This implies that blocking them comes at the cost of losing the network's ability to learn, presenting an impossible tradeoff to the research community.