-
Demystifying optimized prompts in language models
Authors:
Rimon Melamed,
Lucas H. McCabe,
H. Howie Huang
Abstract:
Modern language models (LMs) are not robust to out-of-distribution inputs. Machine generated (``optimized'') prompts can be used to modulate LM outputs and induce specific behaviors while appearing completely uninterpretable. In this work, we investigate the composition of optimized prompts, as well as the mechanisms by which LMs parse and build predictions from optimized prompts. We find that opt…
▽ More
Modern language models (LMs) are not robust to out-of-distribution inputs. Machine generated (``optimized'') prompts can be used to modulate LM outputs and induce specific behaviors while appearing completely uninterpretable. In this work, we investigate the composition of optimized prompts, as well as the mechanisms by which LMs parse and build predictions from optimized prompts. We find that optimized prompts primarily consist of punctuation and noun tokens which are more rare in the training data. Internally, optimized prompts are clearly distinguishable from natural language counterparts based on sparse subsets of the model's activations. Across various families of instruction-tuned models, optimized prompts follow a similar path in how their representations form through the network.
△ Less
Submitted 4 May, 2025;
originally announced May 2025.
-
Network Analysis of U.S. Non-Fatal Opioid-Involved Overdose Journeys, 2018-2023
Authors:
Lucas H. McCabe,
Naoki Masuda,
Shannon Casillas,
Nathan Danneman,
Alen Alic,
Royal Law
Abstract:
We present a nation-wide network analysis of non-fatal opioid-involved overdose journeys in the United States. Leveraging a unique proprietary dataset of Emergency Medical Services incidents, we construct a journey-to-overdose geospatial network capturing nearly half a million opioid-involved overdose events spanning 2018-2023. We analyze the structure and sociological profiles of the nodes, which…
▽ More
We present a nation-wide network analysis of non-fatal opioid-involved overdose journeys in the United States. Leveraging a unique proprietary dataset of Emergency Medical Services incidents, we construct a journey-to-overdose geospatial network capturing nearly half a million opioid-involved overdose events spanning 2018-2023. We analyze the structure and sociological profiles of the nodes, which are counties or their equivalents, characterize the distribution of overdose journey lengths, and investigate changes in the journey network between 2018 and 2023. Our findings include that authority and hub nodes identified by the HITS algorithm tend to be located in urban areas and involved in overdose journeys with particularly long geographical distances.
△ Less
Submitted 4 August, 2024; v1 submitted 6 February, 2024;
originally announced February 2024.
-
Prompts have evil twins
Authors:
Rimon Melamed,
Lucas H. McCabe,
Tanay Wakhare,
Yejin Kim,
H. Howie Huang,
Enric Boix-Adsera
Abstract:
We discover that many natural-language prompts can be replaced by corresponding prompts that are unintelligible to humans but that provably elicit similar behavior in language models. We call these prompts "evil twins" because they are obfuscated and uninterpretable (evil), but at the same time mimic the functionality of the original natural-language prompts (twins). Remarkably, evil twins transfe…
▽ More
We discover that many natural-language prompts can be replaced by corresponding prompts that are unintelligible to humans but that provably elicit similar behavior in language models. We call these prompts "evil twins" because they are obfuscated and uninterpretable (evil), but at the same time mimic the functionality of the original natural-language prompts (twins). Remarkably, evil twins transfer between models. We find these prompts by solving a maximum-likelihood problem which has applications of independent interest.
△ Less
Submitted 6 October, 2024; v1 submitted 12 November, 2023;
originally announced November 2023.
