-
DiffBreak: Is Diffusion-Based Purification Robust?
Authors:
Andre Kassis,
Urs Hengartner,
Yaoliang Yu
Abstract:
Diffusion-based purification (DBP) has become a cornerstone defense against adversarial examples (AEs), regarded as robust due to its use of diffusion models (DMs) that project AEs onto the natural data manifold. We refute this core claim, theoretically proving that gradient-based attacks effectively target the DM rather than the classifier, causing DBP's outputs to align with adversarial distribu…
▽ More
Diffusion-based purification (DBP) has become a cornerstone defense against adversarial examples (AEs), regarded as robust due to its use of diffusion models (DMs) that project AEs onto the natural data manifold. We refute this core claim, theoretically proving that gradient-based attacks effectively target the DM rather than the classifier, causing DBP's outputs to align with adversarial distributions. This prompts a reassessment of DBP's robustness, attributing it to two critical flaws: incorrect gradients and inappropriate evaluation protocols that test only a single random purification of the AE. We show that with proper accounting for stochasticity and resubmission risk, DBP collapses. To support this, we introduce DiffBreak, the first reliable toolkit for differentiation through DBP, eliminating gradient flaws that previously further inflated robustness estimates. We also analyze the current defense scheme used for DBP where classification relies on a single purification, pinpointing its inherent invalidity. We provide a statistically grounded majority-vote (MV) alternative that aggregates predictions across multiple purified copies, showing partial but meaningful robustness gain. We then propose a novel adaptation of an optimization method against deepfake watermarking, crafting systemic perturbations that defeat DBP even under MV, challenging DBP's viability.
△ Less
Submitted 23 May, 2025; v1 submitted 25 November, 2024;
originally announced November 2024.
-
UnMarker: A Universal Attack on Defensive Image Watermarking
Authors:
Andre Kassis,
Urs Hengartner
Abstract:
Reports regarding the misuse of Generative AI (GenAI) to create deepfakes are frequent. Defensive watermarking enables GenAI providers to hide fingerprints in their images and use them later for deepfake detection. Yet, its potential has not been fully explored. We present UnMarker -- the first practical universal attack on defensive watermarking. Unlike existing attacks, UnMarker requires no dete…
▽ More
Reports regarding the misuse of Generative AI (GenAI) to create deepfakes are frequent. Defensive watermarking enables GenAI providers to hide fingerprints in their images and use them later for deepfake detection. Yet, its potential has not been fully explored. We present UnMarker -- the first practical universal attack on defensive watermarking. Unlike existing attacks, UnMarker requires no detector feedback, no unrealistic knowledge of the watermarking scheme or similar models, and no advanced denoising pipelines that may not be available. Instead, being the product of an in-depth analysis of the watermarking paradigm revealing that robust schemes must construct their watermarks in the spectral amplitudes, UnMarker employs two novel adversarial optimizations to disrupt the spectra of watermarked images, erasing the watermarks. Evaluations against SOTA schemes prove UnMarker's effectiveness. It not only defeats traditional schemes while retaining superior quality compared to existing attacks but also breaks semantic watermarks that alter an image's structure, reducing the best detection rate to $43\%$ and rendering them useless. To our knowledge, UnMarker is the first practical attack on semantic watermarks, which have been deemed the future of defensive watermarking. Our findings show that defensive watermarking is not a viable defense against deepfakes, and we urge the community to explore alternatives.
△ Less
Submitted 22 November, 2024; v1 submitted 14 May, 2024;
originally announced May 2024.
-
On the Feasibility of Fingerprinting Collaborative Robot Network Traffic
Authors:
Cheng Tang,
Diogo Barradas,
Urs Hengartner,
Yue Hu
Abstract:
This study examines privacy risks in collaborative robotics, focusing on the potential for traffic analysis in encrypted robot communications. While previous research has explored low-level command recovery in teleoperation setups, our work investigates high-level motion recovery from script-based control interfaces. We evaluate the efficacy of prominent website fingerprinting techniques (e.g., Ti…
▽ More
This study examines privacy risks in collaborative robotics, focusing on the potential for traffic analysis in encrypted robot communications. While previous research has explored low-level command recovery in teleoperation setups, our work investigates high-level motion recovery from script-based control interfaces. We evaluate the efficacy of prominent website fingerprinting techniques (e.g., Tik-Tok, RF) and their limitations in accurately identifying robotic actions due to their inability to capture detailed temporal relationships. To address this, we introduce a traffic classification approach using signal processing techniques, demonstrating high accuracy in action identification and highlighting the vulnerability of encrypted communications to privacy breaches. Additionally, we explore defenses such as packet padding and timing manipulation, revealing the challenges in balancing traffic analysis resistance with network efficiency. Our findings emphasize the need for continued development of practical defenses in robotic privacy and security.
△ Less
Submitted 7 March, 2025; v1 submitted 11 December, 2023;
originally announced December 2023.
-
Practical Attacks on Voice Spoofing Countermeasures
Authors:
Andre Kassis,
Urs Hengartner
Abstract:
Voice authentication has become an integral part in security-critical operations, such as bank transactions and call center conversations. The vulnerability of automatic speaker verification systems (ASVs) to spoofing attacks instigated the development of countermeasures (CMs), whose task is to tell apart bonafide and spoofed speech. Together, ASVs and CMs form today's voice authentication platfor…
▽ More
Voice authentication has become an integral part in security-critical operations, such as bank transactions and call center conversations. The vulnerability of automatic speaker verification systems (ASVs) to spoofing attacks instigated the development of countermeasures (CMs), whose task is to tell apart bonafide and spoofed speech. Together, ASVs and CMs form today's voice authentication platforms, advertised as an impregnable access control mechanism. We develop the first practical attack on CMs, and show how a malicious actor may efficiently craft audio samples to bypass voice authentication in its strictest form. Previous works have primarily focused on non-proactive attacks or adversarial strategies against ASVs that do not produce speech in the victim's voice. The repercussions of our attacks are far more severe, as the samples we generate sound like the victim, eliminating any chance of plausible deniability. Moreover, the few existing adversarial attacks against CMs mistakenly optimize spoofed speech in the feature space and do not take into account the existence of ASVs, resulting in inferior synthetic audio that fails in realistic settings. We eliminate these obstacles through our key technical contribution: a novel joint loss function that enables mounting advanced adversarial attacks against combined ASV/CM deployments directly in the time domain. Our adversarials achieve concerning black-box success rates against state-of-the-art authentication platforms (up to 93.57\%). Finally, we perform the first targeted, over-telephony-network attack on CMs, bypassing several challenges and enabling various potential threats, given the increased use of voice biometrics in call centers. Our results call into question the security of modern voice authentication systems in light of the real threat of attackers bypassing these measures to gain access to users' most valuable resources.
△ Less
Submitted 30 July, 2021;
originally announced July 2021.
-
Investigating Membership Inference Attacks under Data Dependencies
Authors:
Thomas Humphries,
Simon Oya,
Lindsey Tulloch,
Matthew Rafuse,
Ian Goldberg,
Urs Hengartner,
Florian Kerschbaum
Abstract:
Training machine learning models on privacy-sensitive data has become a popular practice, driving innovation in ever-expanding fields. This has opened the door to new attacks that can have serious privacy implications. One such attack, the Membership Inference Attack (MIA), exposes whether or not a particular data point was used to train a model. A growing body of literature uses Differentially Pr…
▽ More
Training machine learning models on privacy-sensitive data has become a popular practice, driving innovation in ever-expanding fields. This has opened the door to new attacks that can have serious privacy implications. One such attack, the Membership Inference Attack (MIA), exposes whether or not a particular data point was used to train a model. A growing body of literature uses Differentially Private (DP) training algorithms as a defence against such attacks. However, these works evaluate the defence under the restrictive assumption that all members of the training set, as well as non-members, are independent and identically distributed. This assumption does not hold for many real-world use cases in the literature. Motivated by this, we evaluate membership inference with statistical dependencies among samples and explain why DP does not provide meaningful protection (the privacy parameter $ε$ scales with the training set size $n$) in this more general case. We conduct a series of empirical evaluations with off-the-shelf MIAs using training sets built from real-world data showing different types of dependencies among samples. Our results reveal that training set dependencies can severely increase the performance of MIAs, and therefore assuming that data samples are statistically independent can significantly underestimate the performance of MIAs.
△ Less
Submitted 14 June, 2023; v1 submitted 22 October, 2020;
originally announced October 2020.
-
Two Novel Defenses against Motion-Based Keystroke Inference Attacks
Authors:
Yihang Song,
Madhur Kukreti,
Rahul Rawat,
Urs Hengartner
Abstract:
Nowadays smartphones come embedded with multiple motion sensors, such as an accelerometer, a gyroscope and an orientation sensor. With these sensors, apps can gather more information and therefore provide end users with more functionality. However, these sensors also introduce the potential risk of leaking a user's private information because apps can access these sensors without requiring securit…
▽ More
Nowadays smartphones come embedded with multiple motion sensors, such as an accelerometer, a gyroscope and an orientation sensor. With these sensors, apps can gather more information and therefore provide end users with more functionality. However, these sensors also introduce the potential risk of leaking a user's private information because apps can access these sensors without requiring security permissions. By monitoring a device's motion, a malicious app may be able to infer sensitive information about the owner of the device. For example, related work has shown that sensitive information entered by a user on a device's touchscreen, such as numerical PINs or passwords, can be inferred from accelerometer and gyroscope data.
In this paper, we study these motion-based keystroke inference attacks to determine what information they need to succeed. Based on this study, we propose two novel approaches to defend against keystroke inference attacks: 1) Reducing sensor data accuracy; 2) Random keyboard layout generation. We present the design and the implementation of these two defences on the Android platform and show how they significantly reduce the accuracy of keystroke inference attacks. We also conduct multiple user studies to evaluate the usability and feasibility of these two defences. Finally, we determine the impact of the defences on apps that have legitimate reasons to access motion sensors and show that the impact is negligible.
△ Less
Submitted 28 October, 2014;
originally announced October 2014.
