#040 – Adversarial Examples (Dr. Nicholas Carlini, Dr. Wieland Brendel, Florian Tramèr)

Pod version; https://anchor.fm/machinelearningstreettalk/episodes/040—Adversarial-Examples-Dr–Nicholas-Carlini–Dr–Wieland-Brendel–Florian-Tramr-epo5qr

Adversarial examples have attracted significant attention in machine learning, but the reasons for their existence and pervasiveness remain unclear. there’s good reason to believe neural networks look at very different features than we would have expected. As articulated in the 2019 “features not bugs” paper Adversarial examples can be directly attributed to the presence of non-robust features: features derived from patterns in the data distribution that are highly predictive, yet brittle and incomprehensible to humans.

Adversarial examples don’t just affect deep learning models. A cottage industry has sprung up around Threat Modeling in AI and ML Systems and their dependencies. Joining us this evening are some of currently leading researchers in adversarial examples;

Florian Tramèr – A fifth year PhD student in Computer Science at Stanford University

Dr. Wieland Brendel – Machine Learning Researcher at the University of Tübingen & Co-Founder of layer7.ai

Dr. Nicholas Carlini – Research scientist at Google Brain working in that exciting space between machine learning and computer security.

We really hope you enjoy the conversation, remember to subscribe!

Yannic Intro [00:00:00]
Tim Intro [00:04:07]
Threat Taxonomy [00:09:00]
Main show intro [00:11:30]
Whats wrong with Neural Networks? [00:14:52]
The role of memorization [00:19:51]
Anthropomorphization of models [00:22:42]
Whats the harm really though / focusing on actual ML security risks [00:27:03]
Shortcut learning / OOD generalization [00:36:18]
Human generalization [00:40:11]
An existential problem in DL getting the models to learn what we want? [00:41:39]
Defenses to adversarial examples [00:47:15]
What if we had all the data and the labels? Still problems? [00:54:28]
Defenses are easily broken [01:00:24]
Self deception in academia [01:06:46]
ML Security [01:28:15]

[1802.00420] Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

[2006.11440] Using Learning Dynamics to Explore the Role of Implicit Regularization in Adversarial Examples

[1905.02175] Adversarial Examples Are Not Bugs, They Are Features

[2004.07780] Shortcut Learning in Deep Neural Networks

[1811.12231] ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

[1902.06705] On Evaluating Adversarial Robustness

[2012.07805] Extracting Training Data from Large Language Models

[1811.03194] AdVersarial: Perceptual Ad Blocking meets Adversarial Machine Learning

[2002.04599] Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations [ICML, 2020]

[2002.08347] On Adaptive Attacks to Adversarial Example Defenses [NeurIPS, 2020]

[Threat Modeling AI/ML Systems and Dependencies]

#machinelearning #deeplearning

YouTube Source for this AI Video

AI video(s) you might be interested in …