-
TrollHunter [Evader]: Automated Detection [Evasion] of Twitter Trolls During the COVID-19 Pandemic
Authors:
Peter Jachim,
Filipo Sharevski,
Paige Treebridge
Abstract:
This paper presents TrollHunter, an automated reasoning mechanism we used to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. Trolls, poised to disrupt the online discourse and spread disinformation, quickly seized the absence of a credible response to COVID-19 and created a COVID-19 infodemic by promulgating dubious content on Twitter. To counter the COVID-19 infodemic, the TrollH…
▽ More
This paper presents TrollHunter, an automated reasoning mechanism we used to hunt for trolls on Twitter during the COVID-19 pandemic in 2020. Trolls, poised to disrupt the online discourse and spread disinformation, quickly seized the absence of a credible response to COVID-19 and created a COVID-19 infodemic by promulgating dubious content on Twitter. To counter the COVID-19 infodemic, the TrollHunter leverages a unique linguistic analysis of a multi-dimensional set of Twitter content features to detect whether or not a tweet was meant to troll. TrollHunter achieved 98.5% accuracy, 75.4% precision and 69.8% recall over a dataset of 1.3 million tweets. Without a final resolution of the pandemic in sight, it is unlikely that the trolls will go away, although they might be forced to evade automated hunting. To explore the plausibility of this strategy, we developed and tested an adversarial machine learning mechanism called TrollHunter-Evader. TrollHunter-Evader employs a Test Time Evasion (TTE) approach in a combination with a Markov chain-based mechanism to recycle originally trolling tweets. The recycled tweets were able to achieve a remarkable 40% decrease in the TrollHunter's ability to correctly identify trolling tweets. Because the COVID-19 infodemic could have a harmful impact on the COVID-19 pandemic, we provide an elaborate discussion about the implications of employing adversarial machine learning to evade Twitter troll hunts.
△ Less
Submitted 6 December, 2020; v1 submitted 4 December, 2020;
originally announced December 2020.
-
Beyond Trolling: Malware-Induced Misperception Attacks on Polarized Facebook Discourse
Authors:
Filipo Sharevski,
Paige Treebridge,
Peter Jachim,
Audrey Li,
Adam Babin,
Jessica Westbrook
Abstract:
Social media trolling is a powerful tactic to manipulate public opinion on issues with a high moral component. Troll farms, as evidenced in the past, created fabricated content to provoke or silence people to share their opinion on social media during the US presidential election in 2016. In this paper, we introduce an alternate way of provoking or silencing social media discourse by manipulating…
▽ More
Social media trolling is a powerful tactic to manipulate public opinion on issues with a high moral component. Troll farms, as evidenced in the past, created fabricated content to provoke or silence people to share their opinion on social media during the US presidential election in 2016. In this paper, we introduce an alternate way of provoking or silencing social media discourse by manipulating how users perceive authentic content. This manipulation is performed by man-in-the-middle malware that covertly rearranges the linguistic content of an authentic social media post and comments. We call this attack Malware-Induced Misperception (MIM) because the goal is to socially engineer spiral-of-silence conditions on social media by inducing perception. We conducted experimental tests in controlled settings (N = 311) where a malware covertly altered selected words in a Facebook post about the freedom of political expression on college campuses. The empirical results (1) confirm the previous findings about the presence of the spiral-of-silence effect on social media; and (2) demonstrate that inducing misperception is an effective tactic to silence or provoke targeted users on Facebook to express their opinion on a polarizing political issue.
△ Less
Submitted 10 February, 2020;
originally announced February 2020.
-
Meet Malexa, Alexa's Malicious Twin: Malware-Induced Misperception Through Intelligent Voice Assistants
Authors:
Filipo Sharevski,
Paige Treebridge,
Peter Jachim,
Audrey Li,
Adam Babin,
Jessica Westbrook
Abstract:
This paper reports the findings of a study where users (N=220) interacted with Malexa, Alexa's malicious twin. Malexa is an intelligent voice assistant with a simple and seemingly harmless third-party skill that delivers news briefings to users. The twist, however, is that Malexa covertly rewords these briefings to intentionally introduce misperception about the reported events. This covert reword…
▽ More
This paper reports the findings of a study where users (N=220) interacted with Malexa, Alexa's malicious twin. Malexa is an intelligent voice assistant with a simple and seemingly harmless third-party skill that delivers news briefings to users. The twist, however, is that Malexa covertly rewords these briefings to intentionally introduce misperception about the reported events. This covert rewording is referred to as a Malware-Induced Misperception (MIM) attack. It differs from squatting or invocation hijacking attacks in that it is focused on manipulating the "content" delivered through a third-party skill instead of the skill's "invocation logic." Malexa, in the study, reworded regulatory briefings to make a government response sound more accidental or lenient than the original news delivered by Alexa. The results show that users who interacted with Malexa perceived that the government was less friendly to working people and more in favor of big businesses. The results also show that Malexa is capable of inducing misperceptions regardless of the user's gender, political ideology or frequency of interaction with intelligent voice assistants. We discuss the implications in the context of using Malexa as a covert "influencer" in people's living or working environments.
△ Less
Submitted 9 February, 2020;
originally announced February 2020.
-
Manipulation of Perceived Politeness in a Web-based Email Discourse Through a Malicious Browser Extension
Authors:
Filipo Sharevski,
Paige Treebridge,
Jessica Westbrook
Abstract:
This paper presents a specific man-in-the-middle exploit: Ambient Tactical Deception (ATD) in online communication, realized via a malicious web browser extension. Extensions manipulate web content in unobtrusive ways as ambient intermediaries of the overall browsing experience. In our previous work, we demonstrated that it is possible to employ tactical deception by making covert changes in the t…
▽ More
This paper presents a specific man-in-the-middle exploit: Ambient Tactical Deception (ATD) in online communication, realized via a malicious web browser extension. Extensions manipulate web content in unobtrusive ways as ambient intermediaries of the overall browsing experience. In our previous work, we demonstrated that it is possible to employ tactical deception by making covert changes in the text content of a web page, regardless of the source. In this work, we investigated the application of ATD in a web-based email discourse where the objective is to manipulate the interpersonal perception without the knowledge of the involved parties. We focus on web-based email text because it is asynchronous and usually revised for clarity and politeness. Previous research has demonstrated that people's perception of politeness in online communication is based on three factors: the degree of imposition, the power of the receiver over the sender, and the social distance between them. We interviewed participants about their perception of these factors to establish the plausibility of ATD for email discourse. The results indicate that by covertly altering the politeness strategy in an email, it is possible for an ATD attacker to manipulate the receiver's perception on all of the politeness factors. Our findings support the Brown and Levinson's politeness theory and Walther's hyperpersonal model of email communication.
△ Less
Submitted 24 December, 2019; v1 submitted 30 August, 2019;
originally announced August 2019.
