-
Designing VR Simulation System for Clinical Communication Training with LLMs-Based Embodied Conversational Agents
Authors:
Xiuqi Tommy Zhu,
Heidi Cheerman,
Minxin Cheng,
Sheri Kiami,
Leanne Chukoskie,
Eileen McGivney
Abstract:
VR simulation in Health Professions (HP) education demonstrates huge potential, but fixed learning content with little customization limits its application beyond lab environments. To address these limitations in the context of VR for patient communication training, we conducted a user-centered study involving semi-structured interviews with advanced HP students to understand their challenges in c…
▽ More
VR simulation in Health Professions (HP) education demonstrates huge potential, but fixed learning content with little customization limits its application beyond lab environments. To address these limitations in the context of VR for patient communication training, we conducted a user-centered study involving semi-structured interviews with advanced HP students to understand their challenges in clinical communication training and perceptions of VR-based solutions. From this, we derived design insights emphasizing the importance of realistic scenarios, simple interactions, and unpredictable dialogues. Building on these insights, we developed the Virtual AI Patient Simulator (VAPS), a novel VR system powered by Large Language Models (LLMs) and Embodied Conversational Agents (ECAs), supporting dynamic and customizable patient interactions for immersive learning. We also provided an example of how clinical professors could use user-friendly design forms to create personalized scenarios that align with course objectives in VAPS and discuss future implications of integrating AI-driven technologies into VR education.
△ Less
Submitted 3 March, 2025;
originally announced March 2025.
-
Approximately Jumping Towards the Origin
Authors:
Alex Albors,
François Clément,
Shosuke Kiami,
Braeden Sodt,
Ding Yifan,
Tony Zeng
Abstract:
Given an initial point $x_0 \in \mathbb{R}^d$ and a sequence of vectors $v_1, v_2, \dots$ in $\mathbb{R}^d$, we define a greedy sequence by setting $x_{n} = x_{n-1} \pm v_n$ where the sign is chosen so as to minimize $\|x_n\|$. We prove that if the vectors $v_i$ are chosen uniformly at random from $\mathbb{S}^{d-1}$ then elements of the sequence are, on average, approximately at distance…
▽ More
Given an initial point $x_0 \in \mathbb{R}^d$ and a sequence of vectors $v_1, v_2, \dots$ in $\mathbb{R}^d$, we define a greedy sequence by setting $x_{n} = x_{n-1} \pm v_n$ where the sign is chosen so as to minimize $\|x_n\|$. We prove that if the vectors $v_i$ are chosen uniformly at random from $\mathbb{S}^{d-1}$ then elements of the sequence are, on average, approximately at distance $\|x_n\| \sim \sqrt{πd/8}$ from the origin. We show that the sequence $(\|x_n\|)_{n=1}^{\infty}$ has an invariant measure $π_d$ depending only on $d$ and we determine its mean and study its decay for all $d$. We also investigate a completely deterministic example in $d=2$ where the $v_n$ are derived from the van der Corput sequence. Several additional examples are considered.
△ Less
Submitted 5 December, 2024;
originally announced December 2024.
-
First-order hold lossless convexification: theoretical guarantees for discrete-time optimal control problems
Authors:
Shosuke Kiami
Abstract:
Lossless Convexification (LCvx) is a clever trick that transforms a class of nonconvex optimal control problems (where the nonconvexity arises from a lower bound on the control norm) into equivalent convex problems via convex relaxations, the goal being to solve these problems efficiently via polynomial-time numerical solvers. However, to solve these infinite-dimensional problems in practice, they…
▽ More
Lossless Convexification (LCvx) is a clever trick that transforms a class of nonconvex optimal control problems (where the nonconvexity arises from a lower bound on the control norm) into equivalent convex problems via convex relaxations, the goal being to solve these problems efficiently via polynomial-time numerical solvers. However, to solve these infinite-dimensional problems in practice, they must first be converted into finite-dimensional problems, and it remains an open area of research to ensure the theoretical guarantees of LCvx are maintained across this discretization step. Prior work has proven guarantees for zero-order hold control parameterization. In this work, we extend these results to the more general, and practically useful, first-order hold control parameterization. We first show that under mild assumptions, we are guaranteed a solution that violates our nonconvex constraint at no more than $n_x + 1$ vertices in our discretized trajectory (where $n_x$ is the dimension of our state-space). Then, we discuss an algorithm that, for a specific case of problems, finds a solution where our nonconvex constraint is violated along no more than $2n_x + 2$ edges in at most $\lceil \log_2 ((ρ_{\max} - ρ_{\min}) / \varepsilon_ρ) \rceil + 1$ calls to our solver (where $[ρ_{\min}, ρ_{\max}]$ represent the bounds on our control norm and $\varepsilon_ρ$ is some desired suboptimality tolerance). Finally, we provide numerical results demonstrating the effectiveness of our proposed method.
△ Less
Submitted 26 November, 2024;
originally announced November 2024.
-
Finding high posterior density phylogenies by systematically extending a directed acyclic graph
Authors:
Chris Jennings-Shaffer,
David H Rich,
Matthew Macaulay,
Michael D Karcher,
Tanvi Ganapathy,
Shosuke Kiami,
Anna Kooperberg,
Cheng Zhang,
Marc A Suchard,
Frederick A Matsen IV
Abstract:
Bayesian phylogenetics typically estimates a posterior distribution, or aspects thereof, using Markov chain Monte Carlo methods. These methods integrate over tree space by applying local rearrangements to move a tree through its space as a random walk. Previous work explored the possibility of replacing this random walk with a systematic search, but was quickly overwhelmed by the large number of p…
▽ More
Bayesian phylogenetics typically estimates a posterior distribution, or aspects thereof, using Markov chain Monte Carlo methods. These methods integrate over tree space by applying local rearrangements to move a tree through its space as a random walk. Previous work explored the possibility of replacing this random walk with a systematic search, but was quickly overwhelmed by the large number of probable trees in the posterior distribution. In this paper we develop methods to sidestep this problem using a recently introduced structure called the subsplit directed acyclic graph (sDAG). This structure can represent many trees at once, and local rearrangements of trees translate to methods of enlarging the sDAG. Here we propose two methods of introducing, ranking, and selecting local rearrangements on sDAGs to produce a collection of trees with high posterior density. One of these methods successfully recovers the set of high posterior density trees across a range of data sets. However, we find that a simpler strategy of aggregating trees into an sDAG in fact is computationally faster and returns a higher fraction of probable trees.
△ Less
Submitted 18 November, 2024; v1 submitted 13 November, 2024;
originally announced November 2024.
-
GenAug: Retargeting behaviors to unseen situations via Generative Augmentation
Authors:
Zoey Chen,
Sho Kiami,
Abhishek Gupta,
Vikash Kumar
Abstract:
Robot learning methods have the potential for widespread generalization across tasks, environments, and objects. However, these methods require large diverse datasets that are expensive to collect in real-world robotics settings. For robot learning to generalize, we must be able to leverage sources of data or priors beyond the robot's own experience. In this work, we posit that image-text generati…
▽ More
Robot learning methods have the potential for widespread generalization across tasks, environments, and objects. However, these methods require large diverse datasets that are expensive to collect in real-world robotics settings. For robot learning to generalize, we must be able to leverage sources of data or priors beyond the robot's own experience. In this work, we posit that image-text generative models, which are pre-trained on large corpora of web-scraped data, can serve as such a data source. We show that despite these generative models being trained on largely non-robotics data, they can serve as effective ways to impart priors into the process of robot learning in a way that enables widespread generalization. In particular, we show how pre-trained generative models can serve as effective tools for semantically meaningful data augmentation. By leveraging these pre-trained models for generating appropriate "semantic" data augmentations, we propose a system GenAug that is able to significantly improve policy generalization. We apply GenAug to tabletop manipulation tasks, showing the ability to re-target behavior to novel scenarios, while only requiring marginal amounts of real-world data. We demonstrate the efficacy of this system on a number of object manipulation problems in the real world, showing a 40% improvement in generalization to novel scenes and objects.
△ Less
Submitted 22 February, 2023; v1 submitted 13 February, 2023;
originally announced February 2023.