Seminar: The Good, the Bad and the Ugly: Watermarks, Transferable Attacks and Adversarial Defenses by Dr Sai Ganesh Nagarajan

Abstract: 

Suppose a nonprofit organization plans to open-source a classifier f, but wants to detect its use by embedding a watermark directly into the model. Alice is tasked with creating this watermark. Bob aims to make f, adversarially robust, i.e., to ensure that it is hard to find queries that appear unsuspicious but cause f to make mistakes. Both face challenges: Alice struggles to create a watermark that cannot be removed, and Bob’s defenses become increasingly complex. They realized that their challenges are two sides of the same coin: the impossibility of one task might guarantee the success of the other. 

In this talk, we formally study the aforementioned situation through the lens of interactive protocols. In doing so, we will show that for almost every discriminative learning task, at least one of the two — a watermark or an adversarial defense — exists. The "almost" refers to the fact that we also identify a third, counterintuitive but necessary option, i.e., a scheme we call a transferable attack. By transferable attack, we refer to an efficient algorithm computing queries that look indistinguishable from the data distribution and fool all efficient defenders. Finally, we prove the necessity of a transferable attack via a construction that uses a cryptographic tool called homomorphic encryption. 

This is joint work with Grzegorz Gluch (EPFL -> UC Berkeley), Berkant Turan (TU Berlin, Zuse Institute Berlin) and Sebastian Pokutta (TU Berlin, Zuse Institute Berlin). A preliminary version of this work was presented in the "ICML 2024 Workshop on Theoretical Foundations of Foundation Models". For an overview do check our blogpost : https://www.pokutta.com/blog/research/2024/11/08/thegood-the-bad-and-the-ugly.htm

Bio: 

Sai Ganesh Nagarajan is currently a postdoctoral researcher and leader of the Robust and Explainable Learning thrust at the IOL Lab, Zuse Institute Berlin. Previously, he held a postdoctoral position at the EPFL CS Theory Group. He earned his PhD from the Singapore University of Technology and Design, supported by the prestigious President's Graduate Fellowship. 

Additionally, Sai has three years of industry research experience as a research engineer at the Institute for InfoComm Research, Singapore. There, he specialized in spatial-temporal statistical modeling for startups and government agencies, including the Housing Development Board and Ministry of Health of Singapore. Notably, one such project that studies the micro-climatic effects of building designs in Singapore was recognized with the Ministry for National Development's R\&D Award in 2019.