Training adversarially robust sparse networks via Bayesian connectivity sampling

Publikation: Beitrag in Buch/Bericht/KonferenzbandBeitrag in einem Konferenzband

Abstract

Deep neural networks have been shown to be susceptible to adversarial attacks. This lack of adversarial robustness is even more pronounced when models are compressed in order to meet hardware limitations. Hence, if adversarial robustness is an issue, training of sparsely connected networks necessitates considering adversarially robust sparse learning. Motivated by the efficient and stable computational function of the brain in the presence of a highly dynamic synaptic connectivity structure, we propose an intrinsically sparse rewiring approach to train neural networks with state-of-the-art robust learning objectives under high sparsity. Importantly, in contrast to previously proposed pruning techniques, our approach satisfies global connectivity constraints throughout robust optimization, i.e., it does not require dense pre-training followed by pruning. Based on a Bayesian posterior sampling principle, a network rewiring process simultaneously learns the sparse connectivity structure and the robustness-accuracy trade-off based on the adversarial learning objective. Although our networks are sparsely connected throughout the whole training process, our experimental benchmark evaluations show that their performance is superior to recently proposed robustness-aware network pruning methods which start from densely connected networks.
Originalspracheenglisch
TitelProceedings of Machine Learning Research
UntertitelPMLR
Seiten8314-8324
Band139
PublikationsstatusVeröffentlicht - 2021
Veranstaltung38th International Conference on Machine Learning: ICML 2021 - Virtuell
Dauer: 18 Jul 202124 Jul 2021

Konferenz

Konferenz38th International Conference on Machine Learning
KurztitelICML 2021
OrtVirtuell
Zeitraum18/07/2124/07/21

Fields of Expertise

  • Information, Communication & Computing

Fingerprint

Untersuchen Sie die Forschungsthemen von „Training adversarially robust sparse networks via Bayesian connectivity sampling“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren