-
Algorithmic Prompt Generation for Diverse Human-like Teaming and Communication with Large Language Models
Authors:
Siddharth Srikanth,
Varun Bhatt,
Boshen Zhang,
Werner Hager,
Charles Michael Lewis,
Katia P. Sycara,
Aaquib Tabrez,
Stefanos Nikolaidis
Abstract:
Understanding how humans collaborate and communicate in teams is essential for improving human-agent teaming and AI-assisted decision-making. However, relying solely on data from large-scale user studies is impractical due to logistical, ethical, and practical constraints, necessitating synthetic models of multiple diverse human behaviors. Recently, agents powered by Large Language Models (LLMs) h…
▽ More
Understanding how humans collaborate and communicate in teams is essential for improving human-agent teaming and AI-assisted decision-making. However, relying solely on data from large-scale user studies is impractical due to logistical, ethical, and practical constraints, necessitating synthetic models of multiple diverse human behaviors. Recently, agents powered by Large Language Models (LLMs) have been shown to emulate human-like behavior in social settings. But, obtaining a large set of diverse behaviors requires manual effort in the form of designing prompts. On the other hand, Quality Diversity (QD) optimization has been shown to be capable of generating diverse Reinforcement Learning (RL) agent behavior. In this work, we combine QD optimization with LLM-powered agents to iteratively search for prompts that generate diverse team behavior in a long-horizon, multi-step collaborative environment. We first show, through a human-subjects experiment (n=54 participants), that humans exhibit diverse coordination and communication behavior in this domain. We then show that our approach can effectively replicate trends from human teaming data and also capture behaviors that are not easily observed without collecting large amounts of data. Our findings highlight the combination of QD and LLM-powered agents as an effective tool for studying teaming and communication strategies in multi-agent collaboration.
△ Less
Submitted 4 April, 2025;
originally announced April 2025.
-
Beyond the Lens: Quantifying the Impact of Scientific Documentaries through Amazon Reviews
Authors:
Jill Naiman,
Aria Pessianzadeh,
Hanyu Zhao,
AJ Christensen,
Kalina Borkiewicz,
Shriya Srikanth,
Anushka Gami,
Emma Maxwell,
Louisa Zhang,
Sri Nithya Yeragorla,
Rezvaneh Rezapour
Abstract:
Engaging the public with science is critical for a well-informed population. A popular method of scientific communication is documentaries. Once released, it can be difficult to assess the impact of such works on a large scale, due to the overhead required for in-depth audience feedback studies. In what follows, we overview our complementary approach to qualitative studies through quantitative imp…
▽ More
Engaging the public with science is critical for a well-informed population. A popular method of scientific communication is documentaries. Once released, it can be difficult to assess the impact of such works on a large scale, due to the overhead required for in-depth audience feedback studies. In what follows, we overview our complementary approach to qualitative studies through quantitative impact and sentiment analysis of Amazon reviews for several scientific documentaries. In addition to developing a novel impact category taxonomy for this analysis, we release a dataset containing 1296 human-annotated sentences from 1043 Amazon reviews for six movies created in whole or part by the Advanced Visualization Lab (AVL). This interdisciplinary team is housed at the National Center for Supercomputing Applications and consists of visualization designers who focus on cinematic presentations of scientific data. Using this data, we train and evaluate several machine learning and large language models, discussing their effectiveness and possible generalizability for documentaries beyond those focused on for this work. Themes are also extracted from our annotated dataset which, along with our large language model analysis, demonstrate a measure of the ability of scientific documentaries to engage with the public.
△ Less
Submitted 4 March, 2025; v1 submitted 12 February, 2025;
originally announced February 2025.
-
Evaluating the Usability of LLMs in Threat Intelligence Enrichment
Authors:
Sanchana Srikanth,
Mohammad Hasanuzzaman,
Farah Tasnur Meem
Abstract:
Large Language Models (LLMs) have the potential to significantly enhance threat intelligence by automating the collection, preprocessing, and analysis of threat data. However, the usability of these tools is critical to ensure their effective adoption by security professionals. Despite the advanced capabilities of LLMs, concerns about their reliability, accuracy, and potential for generating inacc…
▽ More
Large Language Models (LLMs) have the potential to significantly enhance threat intelligence by automating the collection, preprocessing, and analysis of threat data. However, the usability of these tools is critical to ensure their effective adoption by security professionals. Despite the advanced capabilities of LLMs, concerns about their reliability, accuracy, and potential for generating inaccurate information persist. This study conducts a comprehensive usability evaluation of five LLMs ChatGPT, Gemini, Cohere, Copilot, and Meta AI focusing on their user interface design, error handling, learning curve, performance, and integration with existing tools in threat intelligence enrichment. Utilizing a heuristic walkthrough and a user study methodology, we identify key usability issues and offer actionable recommendations for improvement. Our findings aim to bridge the gap between LLM functionality and user experience, thereby promoting more efficient and accurate threat intelligence practices by ensuring these tools are user-friendly and reliable.
△ Less
Submitted 23 September, 2024;
originally announced September 2024.
-
Symmetric Models for Visual Force Policy Learning
Authors:
Colin Kohler,
Anuj Shrivatsav Srikanth,
Eshan Arora,
Robert Platt
Abstract:
While it is generally acknowledged that force feedback is beneficial to robotic control, applications of policy learning to robotic manipulation typically only leverage visual feedback. Recently, symmetric neural models have been used to significantly improve the sample efficiency and performance of policy learning across a variety of robotic manipulation domains. This paper explores an applicatio…
▽ More
While it is generally acknowledged that force feedback is beneficial to robotic control, applications of policy learning to robotic manipulation typically only leverage visual feedback. Recently, symmetric neural models have been used to significantly improve the sample efficiency and performance of policy learning across a variety of robotic manipulation domains. This paper explores an application of symmetric policy learning to visual-force problems. We present Symmetric Visual Force Learning (SVFL), a novel method for robotic control which leverages visual and force feedback. We demonstrate that SVFL can significantly outperform state of the art baselines for visual force learning and report several interesting empirical findings related to the utility of learning force feedback control policies in both general manipulation tasks and scenarios with low visual acuity.
△ Less
Submitted 28 August, 2023;
originally announced August 2023.
-
Fast Adaptation of Manipulator Trajectories to Task Perturbation By Differentiating through the Optimal Solution
Authors:
Shashank Srikanth,
Mithun Babu,
Houman Masnavi,
Arun Kumar Singh,
Karl Kruusamäe,
K. Madhava Krishna
Abstract:
Joint space trajectory optimization under end-effector task constraints leads to a challenging non-convex problem. Thus, a real-time adaptation of prior computed trajectories to perturbation in task constraints often becomes intractable. Existing works use the so-called warm-starting of trajectory optimization to improve computational performance. We present a fundamentally different approach that…
▽ More
Joint space trajectory optimization under end-effector task constraints leads to a challenging non-convex problem. Thus, a real-time adaptation of prior computed trajectories to perturbation in task constraints often becomes intractable. Existing works use the so-called warm-starting of trajectory optimization to improve computational performance. We present a fundamentally different approach that relies on deriving analytical gradients of the optimal solution with respect to the task constraint parameters. This gradient map characterizes the direction in which the prior computed joint trajectories need to be deformed to comply with the new task constraints. Subsequently, we develop an iterative line-search algorithm for computing the scale of deformation. Our algorithm provides near real-time adaptation of joint trajectories for a diverse class of task perturbations such as (i) changes in initial and final joint configurations of end-effector orientation-constrained trajectories and (ii) changes in end-effector goal or way-points under end-effector orientation constraints. We relate each of these examples to real-world applications ranging from learning from demonstration to obstacle avoidance. We also show that our algorithm produces trajectories with quality similar to what one would obtain by solving the trajectory optimization from scratch with warm-start initialization. But most importantly, our algorithm achieves a worst-case speed-up of 160x over the latter approach.
△ Less
Submitted 1 November, 2020;
originally announced November 2020.
-
Driving The Last Mile: Characterizing and Understanding Distracted Driving Posts on Social Networks
Authors:
Hemank Lamba,
Shashank Srikanth,
Dheeraj Reddy Pailla,
Shwetanshu Singh,
Karandeep Juneja,
Ponnurangam Kumaraguru
Abstract:
In 2015, 391,000 people were injured due to distracted driving in the US. One of the major reasons behind distracted driving is the use of cell-phones, accounting for 14% of fatal crashes. Social media applications have enabled users to stay connected, however, the use of such applications while driving could have serious repercussions -- often leading the user to be distracted from the road and e…
▽ More
In 2015, 391,000 people were injured due to distracted driving in the US. One of the major reasons behind distracted driving is the use of cell-phones, accounting for 14% of fatal crashes. Social media applications have enabled users to stay connected, however, the use of such applications while driving could have serious repercussions -- often leading the user to be distracted from the road and ending up in an accident. In the context of impression management, it has been discovered that individuals often take a risk (such as teens smoking cigarettes, indulging in narcotics, and participating in unsafe sex) to improve their social standing. Therefore, viewing the phenomena of posting distracted driving posts under the lens of self-presentation, it can be hypothesized that users often indulge in risk-taking behavior on social media to improve their impression among their peers. In this paper, we first try to understand the severity of such social-media-based distractions by analyzing the content posted on a popular social media site where the user is driving and is also simultaneously creating content. To this end, we build a deep learning classifier to identify publicly posted content on social media that involves the user driving. Furthermore, a framework proposed to understand factors behind voluntary risk-taking activity observes that younger individuals are more willing to perform such activities, and men (as opposed to women) are more inclined to take risks. Grounding our observations in this framework, we test these hypotheses on 173 cities across the world. We conduct spatial and temporal analysis on a city-level and understand how distracted driving content posting behavior changes due to varied demographics. We discover that the factors put forth by the framework are significant in estimating the extent of such behavior.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
Is change the only constant? Profile change perspective on #LokSabhaElections2019
Authors:
Kumari Neha,
Shashank Srikanth,
Sonali Singhal,
Shwetanshu Singh,
Arun Balaji Buduru,
Ponnurangam Kumaraguru
Abstract:
Users on Twitter are identified with the help of their profile attributes that consists of username, display name, profile image, to name a few. The profile attributes that users adopt can reflect their interests, belief, or thematic inclinations. Literature has proposed the implications and significance of profile attribute change for a random population of users. However, the use of profile attr…
▽ More
Users on Twitter are identified with the help of their profile attributes that consists of username, display name, profile image, to name a few. The profile attributes that users adopt can reflect their interests, belief, or thematic inclinations. Literature has proposed the implications and significance of profile attribute change for a random population of users. However, the use of profile attribute for endorsements and to start a movement have been under-explored. In this work, we consider #LokSabhaElections2019 as a movement and perform a large-scale study of the profile of users who actively made changes to profile attributes centered around #LokSabhaElections2019. We collect the profile metadata for 49.4M users for a period of 2 months from April 5, 2019 to June 5, 2019 amid #LokSabhaElections2019. We investigate how the profile changes vary for the influential leaders and their followers over the social movement. We further differentiate the organic and inorganic ways to show the political inclination from the prism of profile changes. We report how the addition of election campaign related keywords lead to spread of behavior contagion and further investigate it with respect to "Chowkidar Movement" in detail.
△ Less
Submitted 22 September, 2019;
originally announced September 2019.
-
Don't cross that stop line: Characterizing Traffic Violations in Metropolitan Cities
Authors:
Shashank Srikanth,
Aanshul Sadaria,
Himanshu Bhatia,
Kanay Gupta,
Pratik Jain,
Ponnurangam Kumaraguru
Abstract:
In modern metropolitan cities, the task of ensuring safe roads is of paramount importance. Automated systems of e-challans (Electronic traffic-violation receipt) are now being deployed across cities to record traffic violations and to issue fines. In the present study, an automated e-challan system established in Ahmedabad (Gujarat, India) has been analyzed for characterizing user behaviour, viola…
▽ More
In modern metropolitan cities, the task of ensuring safe roads is of paramount importance. Automated systems of e-challans (Electronic traffic-violation receipt) are now being deployed across cities to record traffic violations and to issue fines. In the present study, an automated e-challan system established in Ahmedabad (Gujarat, India) has been analyzed for characterizing user behaviour, violation types as well as finding spatial and temporal patterns in the data. We describe a method of collecting e-challan data from the e-challan portal of Ahmedabad traffic police and create a dataset of over 3 million e-challans. The dataset was first analyzed to characterize user behaviour with respect to repeat offenses and fine payment. We demonstrate that a lot of users repeat their offenses (traffic violation) frequently and are less likely to pay fines of higher value. Next, we analyze the data from a spatial and temporal perspective and identify certain spatio-temporal patterns present in our dataset. We find that there is a drastic increase/decrease in the number of e-challans issued during the festival days and identify a few hotspots in the city that have high intensity of traffic violations. In the end, we propose a set of 5 features to model recidivism in traffic violations and train multiple classifiers on our dataset to evaluate the effectiveness of our proposed features. The proposed approach achieves 95% accuracy on the dataset.
△ Less
Submitted 31 January, 2020; v1 submitted 17 September, 2019;
originally announced September 2019.
-
INFER: INtermediate representations for FuturE pRediction
Authors:
Shashank Srikanth,
Junaid Ahmed Ansari,
Karnik Ram R,
Sarthak Sharma,
Krishna Murthy J.,
Madhava Krishna K
Abstract:
In urban driving scenarios, forecasting future trajectories of surrounding vehicles is of paramount importance. While several approaches for the problem have been proposed, the best-performing ones tend to require extremely detailed input representations (eg. image sequences). But, such methods do not generalize to datasets they have not been trained on. We propose intermediate representations tha…
▽ More
In urban driving scenarios, forecasting future trajectories of surrounding vehicles is of paramount importance. While several approaches for the problem have been proposed, the best-performing ones tend to require extremely detailed input representations (eg. image sequences). But, such methods do not generalize to datasets they have not been trained on. We propose intermediate representations that are particularly well-suited for future prediction. As opposed to using texture (color) information, we rely on semantics and train an autoregressive model to accurately predict future trajectories of traffic participants (vehicles) (see fig. above). We demonstrate that using semantics provides a significant boost over techniques that operate over raw pixel intensities/disparities. Uncharacteristic of state-of-the-art approaches, our representations and models generalize to completely different datasets, collected across several cities, and also across countries where people drive on opposite sides of the road (left-handed vs right-handed driving). Additionally, we demonstrate an application of our approach in multi-object tracking (data association). To foster further research in transferrable representations and ensure reproducibility, we release all our code and data.
△ Less
Submitted 25 March, 2019;
originally announced March 2019.
-
A Brief Survey of Non-Residue Based Computational Error Correction
Authors:
Sriseshan Srikanth,
Bobin Deng,
Thomas M. Conte
Abstract:
The idea of computational error correction has been around for over half a century. The motivation has largely been to mitigate unreliable devices, manufacturing defects or harsh environments, primarily as a mandatory measure to preserve reliability, or more recently, as a means to lower energy by allowing soft errors to occasionally creep. While residue codes have shown great promise for this pur…
▽ More
The idea of computational error correction has been around for over half a century. The motivation has largely been to mitigate unreliable devices, manufacturing defects or harsh environments, primarily as a mandatory measure to preserve reliability, or more recently, as a means to lower energy by allowing soft errors to occasionally creep. While residue codes have shown great promise for this purpose, there have been several orthogonal non-residue based techniques. In this article, we provide a high level outline of some of these non-residual approaches.
△ Less
Submitted 9 November, 2016;
originally announced November 2016.
-
Survey on Various Gesture Recognition Techniques for Interfacing Machines Based on Ambient Intelligence
Authors:
Harshith C,
Karthik R. Shastry,
Manoj Ravindran,
M. V. V. N. S. Srikanth,
Naveen Lakshmikhanth
Abstract:
Gesture recognition is mainly apprehensive on analyzing the functionality of human wits. The main goal of gesture recognition is to create a system which can recognize specific human gestures and use them to convey information or for device control. Hand gestures provide a separate complementary modality to speech for expressing ones ideas. Information associated with hand gestures in a conversati…
▽ More
Gesture recognition is mainly apprehensive on analyzing the functionality of human wits. The main goal of gesture recognition is to create a system which can recognize specific human gestures and use them to convey information or for device control. Hand gestures provide a separate complementary modality to speech for expressing ones ideas. Information associated with hand gestures in a conversation is degree,discourse structure, spatial and temporal structure. The approaches present can be mainly divided into Data-Glove Based and Vision Based approaches. An important face feature point is the nose tip. Since nose is the highest protruding point from the face. Besides that, it is not affected by facial expressions.Another important function of the nose is that it is able to indicate the head pose. Knowledge of the nose location will enable us to align an unknown 3D face with those in a face database. Eye detection is divided into eye position detection and eye contour detection. Existing works in eye detection can be classified into two major categories: traditional image-based passive approaches and the active IR based approaches. The former uses intensity and shape of eyes for detection and the latter works on the assumption that eyes have a reflection under near IR illumination and produce bright/dark pupil effect. The traditional methods can be broadly classified into three categories: template based methods,appearance based methods and feature based methods. The purpose of this paper is to compare various human Gesture recognition systems for interfacing machines directly to human wits without any corporeal media in an ambient environment.
△ Less
Submitted 30 November, 2010;
originally announced December 2010.
