-
Efficient Storage Integrity in Adversarial Settings
Authors:
Quinn Burke,
Ryan Sheatsley,
Yohan Beugin,
Eric Pauley,
Owen Hines,
Michael Swift,
Patrick McDaniel
Abstract:
Storage integrity is essential to systems and applications that use untrusted storage (e.g., public clouds, end-user devices). However, known methods for achieving storage integrity either suffer from high (and often prohibitive) overheads or provide weak integrity guarantees. In this work, we demonstrate a hybrid approach to storage integrity that simultaneously reduces overhead while providing s…
▽ More
Storage integrity is essential to systems and applications that use untrusted storage (e.g., public clouds, end-user devices). However, known methods for achieving storage integrity either suffer from high (and often prohibitive) overheads or provide weak integrity guarantees. In this work, we demonstrate a hybrid approach to storage integrity that simultaneously reduces overhead while providing strong integrity guarantees. Our system, partially asynchronous integrity checking (PAC), allows disk write commitments to be deferred while still providing guarantees around read integrity. PAC delivers a 5.5X throughput and latency improvement over the state of the art, and 85% of the throughput achieved by non-integrity-assuring approaches. In this way, we show that untrusted storage can be used for integrity-critical workloads without meaningfully sacrificing performance.
△ Less
Submitted 9 April, 2025;
originally announced April 2025.
-
On the Robustness Tradeoff in Fine-Tuning
Authors:
Kunyang Li,
Jean-Charles Noirot Ferrand,
Ryan Sheatsley,
Blaine Hoak,
Yohan Beugin,
Eric Pauley,
Patrick McDaniel
Abstract:
Fine-tuning has become the standard practice for adapting pre-trained (upstream) models to downstream tasks. However, the impact on model robustness is not well understood. In this work, we characterize the robustness-accuracy trade-off in fine-tuning. We evaluate the robustness and accuracy of fine-tuned models over 6 benchmark datasets and 7 different fine-tuning strategies. We observe a consist…
▽ More
Fine-tuning has become the standard practice for adapting pre-trained (upstream) models to downstream tasks. However, the impact on model robustness is not well understood. In this work, we characterize the robustness-accuracy trade-off in fine-tuning. We evaluate the robustness and accuracy of fine-tuned models over 6 benchmark datasets and 7 different fine-tuning strategies. We observe a consistent trade-off between adversarial robustness and accuracy. Peripheral updates such as BitFit are more effective for simple tasks--over 75% above the average measured with area under the Pareto frontiers on CIFAR-10 and CIFAR-100. In contrast, fine-tuning information-heavy layers, such as attention layers via Compacter, achieves a better Pareto frontier on more complex tasks--57.5% and 34.6% above the average on Caltech-256 and CUB-200, respectively. Lastly, we observe that robustness of fine-tuning against out-of-distribution data closely tracks accuracy. These insights emphasize the need for robustness-aware fine-tuning to ensure reliable real-world deployments.
△ Less
Submitted 18 March, 2025;
originally announced March 2025.
-
Adversarial Agents: Black-Box Evasion Attacks with Reinforcement Learning
Authors:
Kyle Domico,
Jean-Charles Noirot Ferrand,
Ryan Sheatsley,
Eric Pauley,
Josiah Hanna,
Patrick McDaniel
Abstract:
Reinforcement learning (RL) offers powerful techniques for solving complex sequential decision-making tasks from experience. In this paper, we demonstrate how RL can be applied to adversarial machine learning (AML) to develop a new class of attacks that learn to generate adversarial examples: inputs designed to fool machine learning models. Unlike traditional AML methods that craft adversarial exa…
▽ More
Reinforcement learning (RL) offers powerful techniques for solving complex sequential decision-making tasks from experience. In this paper, we demonstrate how RL can be applied to adversarial machine learning (AML) to develop a new class of attacks that learn to generate adversarial examples: inputs designed to fool machine learning models. Unlike traditional AML methods that craft adversarial examples independently, our RL-based approach retains and exploits past attack experience to improve future attacks. We formulate adversarial example generation as a Markov Decision Process and evaluate RL's ability to (a) learn effective and efficient attack strategies and (b) compete with state-of-the-art AML. On CIFAR-10, our agent increases the success rate of adversarial examples by 19.4% and decreases the median number of victim model queries per adversarial example by 53.2% from the start to the end of training. In a head-to-head comparison with a state-of-the-art image attack, SquareAttack, our approach enables an adversary to generate adversarial examples with 13.1% more success after 5000 episodes of training. From a security perspective, this work demonstrates a powerful new attack vector that uses RL to attack ML models efficiently and at scale.
△ Less
Submitted 3 March, 2025;
originally announced March 2025.
-
Targeting Alignment: Extracting Safety Classifiers of Aligned LLMs
Authors:
Jean-Charles Noirot Ferrand,
Yohan Beugin,
Eric Pauley,
Ryan Sheatsley,
Patrick McDaniel
Abstract:
Alignment in large language models (LLMs) is used to enforce guidelines such as safety. Yet, alignment fails in the face of jailbreak attacks that modify inputs to induce unsafe outputs. In this paper, we present and evaluate a method to assess the robustness of LLM alignment. We observe that alignment embeds a safety classifier in the target model that is responsible for deciding between refusal…
▽ More
Alignment in large language models (LLMs) is used to enforce guidelines such as safety. Yet, alignment fails in the face of jailbreak attacks that modify inputs to induce unsafe outputs. In this paper, we present and evaluate a method to assess the robustness of LLM alignment. We observe that alignment embeds a safety classifier in the target model that is responsible for deciding between refusal and compliance. We seek to extract an approximation of this classifier, called a surrogate classifier, from the LLM. We develop an algorithm for identifying candidate classifiers from subsets of the LLM model. We evaluate the degree to which the candidate classifiers approximate the model's embedded classifier in benign (F1 score) and adversarial (using surrogates in a white-box attack) settings. Our evaluation shows that the best candidates achieve accurate agreement (an F1 score above 80%) using as little as 20% of the model architecture. Further, we find attacks mounted on the surrogate models can be transferred with high accuracy. For example, a surrogate using only 50% of the Llama 2 model achieved an attack success rate (ASR) of 70%, a substantial improvement over attacking the LLM directly, where we only observed a 22% ASR. These results show that extracting surrogate classifiers is a viable (and highly effective) means for modeling (and therein addressing) the vulnerability of aligned models to jailbreaking attacks.
△ Less
Submitted 27 January, 2025;
originally announced January 2025.
-
ParTEETor: A System for Partial Deployments of TEEs within Tor
Authors:
Rachel King,
Quinn Burke,
Yohan Beugin,
Blaine Hoak,
Kunyang Li,
Eric Pauley,
Ryan Sheatsley,
Patrick McDaniel
Abstract:
The Tor anonymity network allows users such as political activists and those under repressive governments to protect their privacy when communicating over the internet. At the same time, Tor has been demonstrated to be vulnerable to several classes of deanonymizing attacks that expose user behavior and identities. Prior work has shown that these threats can be mitigated by leveraging trusted execu…
▽ More
The Tor anonymity network allows users such as political activists and those under repressive governments to protect their privacy when communicating over the internet. At the same time, Tor has been demonstrated to be vulnerable to several classes of deanonymizing attacks that expose user behavior and identities. Prior work has shown that these threats can be mitigated by leveraging trusted execution environments (TEEs). However, previous proposals assume that all relays in the network will be TEE-based-which as a practical matter is unrealistic. In this work, we introduce ParTEETor, a Tor-variant system, which leverages partial deployments of TEEs to thwart known attacks. We study two modes of operation: non-policy and policy. Non-policy mode uses the existing Tor relay selection algorithm to provide users incident security. Policy mode extends the relay selection algorithm to address the classes of attacks by enforcing a specific TEE circuit configuration. We evaluate ParTEETor for security, performance, and privacy. Our evaluation demonstrates that at even a small TEE penetration (e.g., 10% of relays are TEE-based), users can reach performance of Tor today while enforcing a security policy to guarantee protection from at least two classes of attacks. Overall, we find that partial deployments of TEEs can substantially improve the security of Tor, without a significant impact on performance or privacy.
△ Less
Submitted 26 August, 2024;
originally announced August 2024.
-
Characterizing the Modification Space of Signature IDS Rules
Authors:
Ryan Guide,
Eric Pauley,
Yohan Beugin,
Ryan Sheatsley,
Patrick McDaniel
Abstract:
Signature-based Intrusion Detection Systems (SIDSs) are traditionally used to detect malicious activity in networks. A notable example of such a system is Snort, which compares network traffic against a series of rules that match known exploits. Current SIDS rules are designed to minimize the amount of legitimate traffic flagged incorrectly, reducing the burden on network administrators. However,…
▽ More
Signature-based Intrusion Detection Systems (SIDSs) are traditionally used to detect malicious activity in networks. A notable example of such a system is Snort, which compares network traffic against a series of rules that match known exploits. Current SIDS rules are designed to minimize the amount of legitimate traffic flagged incorrectly, reducing the burden on network administrators. However, different use cases than the traditional one--such as researchers studying trends or analyzing modified versions of known exploits--may require SIDSs to be less constrained in their operation. In this paper, we demonstrate that applying modifications to real-world SIDS rules allow for relaxing some constraints and characterizing the performance space of modified rules. We develop an iterative approach for exploring the space of modifications to SIDS rules. By taking the modifications that expand the ROC curve of performance and altering them further, we show how to modify rules in a directed manner. Using traffic collected and identified as benign or malicious from a cloud telescope, we find that the removal of a single component from SIDS rules has the largest impact on the performance space. Effectively modifying SIDS rules to reduce constraints can enable a broader range of detection for various objectives, from increased security to research purposes.
△ Less
Submitted 14 February, 2024;
originally announced February 2024.
-
Systematic Evaluation of Geolocation Privacy Mechanisms
Authors:
Alban Héon,
Ryan Sheatsley,
Quinn Burke,
Blaine Hoak,
Eric Pauley,
Yohan Beugin,
Patrick McDaniel
Abstract:
Location data privacy has become a serious concern for users as Location Based Services (LBSs) have become an important part of their life. It is possible for malicious parties having access to geolocation data to learn sensitive information about the user such as religion or political views. Location Privacy Preserving Mechanisms (LPPMs) have been proposed by previous works to ensure the privacy…
▽ More
Location data privacy has become a serious concern for users as Location Based Services (LBSs) have become an important part of their life. It is possible for malicious parties having access to geolocation data to learn sensitive information about the user such as religion or political views. Location Privacy Preserving Mechanisms (LPPMs) have been proposed by previous works to ensure the privacy of the shared data while allowing the users to use LBSs. But there is no clear view of which mechanism to use according to the scenario in which the user makes use of a LBS. The scenario is the way the user is using a LBS (frequency of reports, number of reports). In this paper, we study the sensitivity of LPPMs on the scenario on which they are used. We propose a framework to systematically evaluate LPPMs by considering an exhaustive combination of LPPMs, attacks and metrics. Using our framework we compare a selection of LPPMs including an improved mechanism that we introduce. By evaluating over a variety of scenarios, we find that the efficacy (privacy, utility, and robustness) of the studied mechanisms is dependent on the scenario: for example the privacy of Planar Laplace geo-indistinguishability is greatly reduced in a continuous scenario. We show that the scenario is essential to consider when choosing an obfuscation mechanism for a given application.
△ Less
Submitted 12 September, 2023;
originally announced September 2023.
-
Securing Cloud File Systems with Trusted Execution
Authors:
Quinn Burke,
Yohan Beugin,
Blaine Hoak,
Rachel King,
Eric Pauley,
Ryan Sheatsley,
Mingli Yu,
Ting He,
Thomas La Porta,
Patrick McDaniel
Abstract:
Cloud file systems offer organizations a scalable and reliable file storage solution. However, cloud file systems have become prime targets for adversaries, and traditional designs are not equipped to protect organizations against the myriad of attacks that may be initiated by a malicious cloud provider, co-tenant, or end-client. Recently proposed designs leveraging cryptographic techniques and tr…
▽ More
Cloud file systems offer organizations a scalable and reliable file storage solution. However, cloud file systems have become prime targets for adversaries, and traditional designs are not equipped to protect organizations against the myriad of attacks that may be initiated by a malicious cloud provider, co-tenant, or end-client. Recently proposed designs leveraging cryptographic techniques and trusted execution environments (TEEs) still force organizations to make undesirable trade-offs, consequently leading to either security, functional, or performance limitations. In this paper, we introduce BFS, a cloud file system that leverages the security capabilities provided by TEEs to bootstrap new security protocols that deliver strong security guarantees, high-performance, and a transparent POSIX-like interface to clients. BFS delivers stronger security guarantees and up to a 2.5X speedup over a state-of-the-art secure file system. Moreover, compared to the industry standard NFS, BFS achieves up to 2.2X speedups across micro-benchmarks and incurs <1X overhead for most macro-benchmark workloads. BFS demonstrates a holistic cloud file system design that does not sacrifice an organizations' security yet can embrace all of the functional and performance advantages of outsourcing.
△ Less
Submitted 2 October, 2024; v1 submitted 29 May, 2023;
originally announced May 2023.
-
Secure IP Address Allocation at Cloud Scale
Authors:
Eric Pauley,
Kyle Domico,
Blaine Hoak,
Ryan Sheatsley,
Quinn Burke,
Yohan Beugin,
Engin Kirda,
Patrick McDaniel
Abstract:
Public clouds necessitate dynamic resource allocation and sharing. However, the dynamic allocation of IP addresses can be abused by adversaries to source malicious traffic, bypass rate limiting systems, and even capture traffic intended for other cloud tenants. As a result, both the cloud provider and their customers are put at risk, and defending against these threats requires a rigorous analysis…
▽ More
Public clouds necessitate dynamic resource allocation and sharing. However, the dynamic allocation of IP addresses can be abused by adversaries to source malicious traffic, bypass rate limiting systems, and even capture traffic intended for other cloud tenants. As a result, both the cloud provider and their customers are put at risk, and defending against these threats requires a rigorous analysis of tenant behavior, adversarial strategies, and cloud provider policies. In this paper, we develop a practical defense for IP address allocation through such an analysis. We first develop a statistical model of cloud tenant deployment behavior based on literature and measurement of deployed systems. Through this, we analyze IP allocation policies under existing and novel threat models. In response to our stronger proposed threat model, we design IP scan segmentation, an IP allocation policy that protects the address pool against adversarial scanning even when an adversary is not limited by number of cloud tenants. Through empirical evaluation on both synthetic and real-world allocation traces, we show that IP scan segmentation reduces adversaries' ability to rapidly allocate addresses, protecting both address space reputation and cloud tenant data. In this way, we show that principled analysis and implementation of cloud IP address allocation can lead to substantial security gains for tenants and their users.
△ Less
Submitted 10 September, 2024; v1 submitted 26 October, 2022;
originally announced October 2022.
-
The Space of Adversarial Strategies
Authors:
Ryan Sheatsley,
Blaine Hoak,
Eric Pauley,
Patrick McDaniel
Abstract:
Adversarial examples, inputs designed to induce worst-case behavior in machine learning models, have been extensively studied over the past decade. Yet, our understanding of this phenomenon stems from a rather fragmented pool of knowledge; at present, there are a handful of attacks, each with disparate assumptions in threat models and incomparable definitions of optimality. In this paper, we propo…
▽ More
Adversarial examples, inputs designed to induce worst-case behavior in machine learning models, have been extensively studied over the past decade. Yet, our understanding of this phenomenon stems from a rather fragmented pool of knowledge; at present, there are a handful of attacks, each with disparate assumptions in threat models and incomparable definitions of optimality. In this paper, we propose a systematic approach to characterize worst-case (i.e., optimal) adversaries. We first introduce an extensible decomposition of attacks in adversarial machine learning by atomizing attack components into surfaces and travelers. With our decomposition, we enumerate over components to create 576 attacks (568 of which were previously unexplored). Next, we propose the Pareto Ensemble Attack (PEA): a theoretical attack that upper-bounds attack performance. With our new attacks, we measure performance relative to the PEA on: both robust and non-robust models, seven datasets, and three extended lp-based threat models incorporating compute costs, formalizing the Space of Adversarial Strategies. From our evaluation we find that attack performance to be highly contextual: the domain, model robustness, and threat model can have a profound influence on attack efficacy. Our investigation suggests that future studies measuring the security of machine learning should: (1) be contextualized to the domain & threat models, and (2) go beyond the handful of known attacks used today.
△ Less
Submitted 6 September, 2023; v1 submitted 9 September, 2022;
originally announced September 2022.
-
Privacy-Preserving Protocols for Smart Cameras and Other IoT Devices
Authors:
Yohan Beugin,
Quinn Burke,
Blaine Hoak,
Ryan Sheatsley,
Eric Pauley,
Gang Tan,
Syed Rafiul Hussain,
Patrick McDaniel
Abstract:
Millions of consumers depend on smart camera systems to remotely monitor their homes and businesses. However, the architecture and design of popular commercial systems require users to relinquish control of their data to untrusted third parties, such as service providers (e.g., the cloud). Third parties therefore can (and in some instances have) access the video footage without the users' knowledg…
▽ More
Millions of consumers depend on smart camera systems to remotely monitor their homes and businesses. However, the architecture and design of popular commercial systems require users to relinquish control of their data to untrusted third parties, such as service providers (e.g., the cloud). Third parties therefore can (and in some instances have) access the video footage without the users' knowledge or consent -- violating the core tenet of user privacy. In this paper, we introduce CaCTUs, a privacy-preserving smart camera system that returns control to the user; the root of trust begins with the user and is maintained through a series of cryptographic protocols designed to support popular features, such as sharing, deleting, and viewing videos live. In so doing, we demonstrate that it is feasible to implement a performant smart-camera system that leverages the convenience of a cloud-based model while retaining the ability to control access to (private) data. We then discuss how our techniques and protocols can also be extended to privacy-preserving designs of other IoT devices recording time series data.
△ Less
Submitted 20 August, 2022;
originally announced August 2022.
-
Measuring and Mitigating the Risk of IP Reuse on Public Clouds
Authors:
Eric Pauley,
Ryan Sheatsley,
Blaine Hoak,
Quinn Burke,
Yohan Beugin,
Patrick McDaniel
Abstract:
Public clouds provide scalable and cost-efficient computing through resource sharing. However, moving from traditional on-premises service management to clouds introduces new challenges; failure to correctly provision, maintain, or decommission elastic services can lead to functional failure and vulnerability to attack. In this paper, we explore a broad class of attacks on clouds which we refer to…
▽ More
Public clouds provide scalable and cost-efficient computing through resource sharing. However, moving from traditional on-premises service management to clouds introduces new challenges; failure to correctly provision, maintain, or decommission elastic services can lead to functional failure and vulnerability to attack. In this paper, we explore a broad class of attacks on clouds which we refer to as cloud squatting. In a cloud squatting attack, an adversary allocates resources in the cloud (e.g., IP addresses) and thereafter leverages latent configuration to exploit prior tenants. To measure and categorize cloud squatting we deployed a custom Internet telescope within the Amazon Web Services us-east-1 region. Using this apparatus, we deployed over 3 million servers receiving 1.5 million unique IP addresses (56% of the available pool) over 101 days beginning in March of 2021. We identified 4 classes of cloud services, 7 classes of third-party services, and DNS as sources of exploitable latent configurations. We discovered that exploitable configurations were both common and in many cases extremely dangerous; we received over 5 million cloud messages, many containing sensitive data such as financial transactions, GPS location, and PII. Within the 7 classes of third-party services, we identified dozens of exploitable software systems spanning hundreds of servers (e.g., databases, caches, mobile applications, and web services). Lastly, we identified 5446 exploitable domains spanning 231 eTLDs-including 105 in the top 10,000 and 23 in the top 1000 popular domains. Through tenant disclosures we have identified several root causes, including (a) a lack of organizational controls, (b) poor service hygiene, and (c) failure to follow best practices. We conclude with a discussion of the space of possible mitigations and describe the mitigations to be deployed by Amazon in response to this study.
△ Less
Submitted 11 April, 2022;
originally announced April 2022.
-
Building a Privacy-Preserving Smart Camera System
Authors:
Yohan Beugin,
Quinn Burke,
Blaine Hoak,
Ryan Sheatsley,
Eric Pauley,
Gang Tan,
Syed Rafiul Hussain,
Patrick McDaniel
Abstract:
Millions of consumers depend on smart camera systems to remotely monitor their homes and businesses. However, the architecture and design of popular commercial systems require users to relinquish control of their data to untrusted third parties, such as service providers (e.g., the cloud). Third parties therefore can (and in some instances have) access the video footage without the users' knowledg…
▽ More
Millions of consumers depend on smart camera systems to remotely monitor their homes and businesses. However, the architecture and design of popular commercial systems require users to relinquish control of their data to untrusted third parties, such as service providers (e.g., the cloud). Third parties therefore can (and in some instances have) access the video footage without the users' knowledge or consent -- violating the core tenet of user privacy. In this paper, we present CaCTUs, a privacy-preserving smart Camera system Controlled Totally by Users. CaCTUs returns control to the user; the root of trust begins with the user and is maintained through a series of cryptographic protocols, designed to support popular features, such as sharing, deleting, and viewing videos live. We show that the system can support live streaming with a latency of 2s at a frame rate of 10fps and a resolution of 480p. In so doing, we demonstrate that it is feasible to implement a performant smart-camera system that leverages the convenience of a cloud-based model while retaining the ability to control access to (private) data.
△ Less
Submitted 23 January, 2022;
originally announced January 2022.
-
On the Robustness of Domain Constraints
Authors:
Ryan Sheatsley,
Blaine Hoak,
Eric Pauley,
Yohan Beugin,
Michael J. Weisman,
Patrick McDaniel
Abstract:
Machine learning is vulnerable to adversarial examples-inputs designed to cause models to perform poorly. However, it is unclear if adversarial examples represent realistic inputs in the modeled domains. Diverse domains such as networks and phishing have domain constraints-complex relationships between features that an adversary must satisfy for an attack to be realized (in addition to any adversa…
▽ More
Machine learning is vulnerable to adversarial examples-inputs designed to cause models to perform poorly. However, it is unclear if adversarial examples represent realistic inputs in the modeled domains. Diverse domains such as networks and phishing have domain constraints-complex relationships between features that an adversary must satisfy for an attack to be realized (in addition to any adversary-specific goals). In this paper, we explore how domain constraints limit adversarial capabilities and how adversaries can adapt their strategies to create realistic (constraint-compliant) examples. In this, we develop techniques to learn domain constraints from data, and show how the learned constraints can be integrated into the adversarial crafting process. We evaluate the efficacy of our approach in network intrusion and phishing datasets and find: (1) up to 82% of adversarial examples produced by state-of-the-art crafting algorithms violate domain constraints, (2) domain constraints are robust to adversarial examples; enforcing constraints yields an increase in model accuracy by up to 34%. We observe not only that adversaries must alter inputs to satisfy domain constraints, but that these constraints make the generation of valid adversarial examples far more challenging.
△ Less
Submitted 7 November, 2021; v1 submitted 18 May, 2021;
originally announced May 2021.
-
Program Analysis of Commodity IoT Applications for Security and Privacy: Challenges and Opportunities
Authors:
Z. Berkay Celik,
Earlence Fernandes,
Eric Pauley,
Gang Tan,
Patrick McDaniel
Abstract:
Recent advances in Internet of Things (IoT) have enabled myriad domains such as smart homes, personal monitoring devices, and enhanced manufacturing. IoT is now pervasive---new applications are being used in nearly every conceivable environment, which leads to the adoption of device-based interaction and automation. However, IoT has also raised issues about the security and privacy of these digita…
▽ More
Recent advances in Internet of Things (IoT) have enabled myriad domains such as smart homes, personal monitoring devices, and enhanced manufacturing. IoT is now pervasive---new applications are being used in nearly every conceivable environment, which leads to the adoption of device-based interaction and automation. However, IoT has also raised issues about the security and privacy of these digitally augmented spaces. Program analysis is crucial in identifying those issues, yet the application and scope of program analysis in IoT remains largely unexplored by the technical community. In this paper, we study privacy and security issues in IoT that require program-analysis techniques with an emphasis on identified attacks against these systems and defenses implemented so far. Based on a study of five IoT programming platforms, we identify the key insights that result from research efforts in both the program analysis and security communities and relate the efficacy of program-analysis techniques to security and privacy issues. We conclude by studying recent IoT analysis systems and exploring their implementations. Through these explorations, we highlight key challenges and opportunities in calibrating for the environments in which IoT systems will be used.
△ Less
Submitted 24 December, 2018; v1 submitted 18 September, 2018;
originally announced September 2018.
