-
Adapting to Evolving Adversaries with Regularized Continual Robust Training
Authors:
Sihui Dai,
Christian Cianfarani,
Arjun Bhagoji,
Vikash Sehwag,
Prateek Mittal
Abstract:
Robust training methods typically defend against specific attack types, such as Lp attacks with fixed budgets, and rarely account for the fact that defenders may encounter new attacks over time. A natural solution is to adapt the defended model to new adversaries as they arise via fine-tuning, a method which we call continual robust training (CRT). However, when implemented naively, fine-tuning on…
▽ More
Robust training methods typically defend against specific attack types, such as Lp attacks with fixed budgets, and rarely account for the fact that defenders may encounter new attacks over time. A natural solution is to adapt the defended model to new adversaries as they arise via fine-tuning, a method which we call continual robust training (CRT). However, when implemented naively, fine-tuning on new attacks degrades robustness on previous attacks. This raises the question: how can we improve the initial training and fine-tuning of the model to simultaneously achieve robustness against previous and new attacks? We present theoretical results which show that the gap in a model's robustness against different attacks is bounded by how far each attack perturbs a sample in the model's logit space, suggesting that regularizing with respect to this logit space distance can help maintain robustness against previous attacks. Extensive experiments on 3 datasets (CIFAR-10, CIFAR-100, and ImageNette) and over 100 attack combinations demonstrate that the proposed regularization improves robust accuracy with little overhead in training time. Our findings and open-source code lay the groundwork for the deployment of models robust to evolving attacks.
△ Less
Submitted 6 February, 2025;
originally announced February 2025.
-
Understanding and Mitigating the Impacts of Differentially Private Census Data on State Level Redistricting
Authors:
Christian Cianfarani,
Aloni Cohen
Abstract:
Data from the Decennial Census is published only after applying a disclosure avoidance system (DAS). Data users were shaken by the adoption of differential privacy in the 2020 DAS, a radical departure from past methods. The change raises the question of whether redistricting law permits, forbids, or requires taking account of the effect of disclosure avoidance. Such uncertainty creates legal risks…
▽ More
Data from the Decennial Census is published only after applying a disclosure avoidance system (DAS). Data users were shaken by the adoption of differential privacy in the 2020 DAS, a radical departure from past methods. The change raises the question of whether redistricting law permits, forbids, or requires taking account of the effect of disclosure avoidance. Such uncertainty creates legal risks for redistricters, as Alabama argued in a lawsuit seeking to prevent the 2020 DAS's deployment. We consider two redistricting settings in which a data user might be concerned about the impacts of privacy preserving noise: drawing equal population districts and litigating voting rights cases. What discrepancies arise if the user does nothing to account for disclosure avoidance? How might the user adapt her analyses to mitigate those discrepancies? We study these questions by comparing the official 2010 Redistricting Data to the 2010 Demonstration Data -- created using the 2020 DAS -- in an analysis of millions of algorithmically generated state legislative redistricting plans. In both settings, we observe that an analyst may come to incorrect conclusions if they do not account for noise. With minor adaptations, though, the underlying policy goals remain achievable: tweaking selection criteria enables a redistricter to draw balanced plans, and illustrative plans can still be used as evidence of the maximum number of majority-minority districts that are possible in a geography. At least for state legislatures, Alabama's claim that differential privacy ``inhibits a State's right to draw fair lines'' appears unfounded.
△ Less
Submitted 10 September, 2024;
originally announced September 2024.
-
Understanding Robust Learning through the Lens of Representation Similarities
Authors:
Christian Cianfarani,
Arjun Nitin Bhagoji,
Vikash Sehwag,
Ben Y. Zhao,
Prateek Mittal,
Haitao Zheng
Abstract:
Representation learning, i.e. the generation of representations useful for downstream applications, is a task of fundamental importance that underlies much of the success of deep neural networks (DNNs). Recently, robustness to adversarial examples has emerged as a desirable property for DNNs, spurring the development of robust training methods that account for adversarial examples. In this paper,…
▽ More
Representation learning, i.e. the generation of representations useful for downstream applications, is a task of fundamental importance that underlies much of the success of deep neural networks (DNNs). Recently, robustness to adversarial examples has emerged as a desirable property for DNNs, spurring the development of robust training methods that account for adversarial examples. In this paper, we aim to understand how the properties of representations learned by robust training differ from those obtained from standard, non-robust training. This is critical to diagnosing numerous salient pitfalls in robust networks, such as, degradation of performance on benign inputs, poor generalization of robustness, and increase in over-fitting. We utilize a powerful set of tools known as representation similarity metrics, across three vision datasets, to obtain layer-wise comparisons between robust and non-robust DNNs with different training procedures, architectural parameters and adversarial constraints. Our experiments highlight hitherto unseen properties of robust representations that we posit underlie the behavioral differences of robust networks. We discover a lack of specialization in robust networks' representations along with a disappearance of `block structure'. We also find overfitting during robust training largely impacts deeper layers. These, along with other findings, suggest ways forward for the design and training of better robust networks.
△ Less
Submitted 15 September, 2022; v1 submitted 20 June, 2022;
originally announced June 2022.
-
"Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real World
Authors:
Emily Wenger,
Max Bronckers,
Christian Cianfarani,
Jenna Cryan,
Angela Sha,
Haitao Zheng,
Ben Y. Zhao
Abstract:
Advances in deep learning have introduced a new wave of voice synthesis tools, capable of producing audio that sounds as if spoken by a target speaker. If successful, such tools in the wrong hands will enable a range of powerful attacks against both humans and software systems (aka machines). This paper documents efforts and findings from a comprehensive experimental study on the impact of deep-le…
▽ More
Advances in deep learning have introduced a new wave of voice synthesis tools, capable of producing audio that sounds as if spoken by a target speaker. If successful, such tools in the wrong hands will enable a range of powerful attacks against both humans and software systems (aka machines). This paper documents efforts and findings from a comprehensive experimental study on the impact of deep-learning based speech synthesis attacks on both human listeners and machines such as speaker recognition and voice-signin systems. We find that both humans and machines can be reliably fooled by synthetic speech and that existing defenses against synthesized speech fall short. These findings highlight the need to raise awareness and develop new protections against synthetic speech for both humans and machines.
△ Less
Submitted 20 September, 2021;
originally announced September 2021.
-
Latest results on quiescent and post-disruption runaway electron mitigation experiments at Frascati Tokamak Upgrade
Authors:
D. Carnevale,
P. Buratti,
M. Baruzzo,
W. Bin,
F. Bombarda,
L. Boncagni,
C. Paz-Soldan,
L. Calacci,
M. Cappelli,
C. Castaldo,
S. Ceccuzzi,
C. Centioli,
C. Cianfarani,
S. Coda,
F. Cordella,
O. D Arcangelo,
J. Decker,
B. Duval,
B. Esposito,
L. Gabellieri,
S. Galeani,
S. Garavaglia,
C. Galperti,
G. Ghillardi,
G. Granucci
, et al. (16 additional authors not shown)
Abstract:
Results from the last FTU campaigns on the deuterium large (wrt FTU volume) pellet REs suppression capability, mainly due to the induced burst MHD activity expelling REs seed are presented for discharges with 0.5 MA and 5.3T. Clear indications of avalanche multiplication of REs following single pellet injection on 0.36 MA flat-top discharges is shown together with quantitative indications of dissi…
▽ More
Results from the last FTU campaigns on the deuterium large (wrt FTU volume) pellet REs suppression capability, mainly due to the induced burst MHD activity expelling REs seed are presented for discharges with 0.5 MA and 5.3T. Clear indications of avalanche multiplication of REs following single pellet injection on 0.36 MA flat-top discharges is shown together with quantitative indications of dissipative effects in terms of critical electrical field increase due to fan-like instabilities. Analysis of large fan-like instabilities on post-disruption RE beams, that seem to be correlated with low electrical field and background density drops, reveal their strong RE energy suppression capability suggesting a new strategy for RE energy suppression controlling large fan instabilities. We demonstrate how such density drops can be induced using modulated ECRH power on post-disruption beams.
△ Less
Submitted 25 May, 2021; v1 submitted 10 May, 2021;
originally announced May 2021.
-
Runaway Electron Control in FTU
Authors:
D. Carnevale,
B. Esposito,
M. Gospodarczyk,
L. Boncagni,
M. Sassano,
S. Galeani,
D. Marocco,
L. Panaccione,
O. Tudisco,
W. Bin,
C. Cianfarani,
G. Ferrò,
G. Granucci,
A. Gabrielli,
C. Maddaluno,
J. R. Martìn-Solìs,
Z. Popovic,
F. Martinelli,
G. Pucella,
G. Ramogida,
M. Riva,
FTU Team
Abstract:
Experimental results on the position and current control of disruption generated runaway electrons (RE) in FTU are presented. A scanning interferometer diagnostic has been used to analyze the time evolution of the RE beam radial position and its instabilities. Correspondence of the interferometer time traces, radial profile reconstructed via magnetic measurements and fission chamber signals are di…
▽ More
Experimental results on the position and current control of disruption generated runaway electrons (RE) in FTU are presented. A scanning interferometer diagnostic has been used to analyze the time evolution of the RE beam radial position and its instabilities. Correspondence of the interferometer time traces, radial profile reconstructed via magnetic measurements and fission chamber signals are discussed. New RE control algorithms, which define in real-time updated plasma current and position references, have been tested in two experimental scenarios featuring disruption generated RE plateaus. Comparative studies among 52 discharges with disruption generated RE beam plateaus are presented in order to assess the effectiveness of the proposed control strategies as the RE beam interaction with the plasma facing components is reduced while the current is ramped-down.
△ Less
Submitted 11 December, 2015; v1 submitted 22 August, 2015;
originally announced August 2015.
