-
Co-Designing Augmented Reality Tools for High-Stakes Clinical Teamwork
Authors:
Angelique Taylor,
Tauhid Tanjim,
Huajie Cao,
Jalynn Blu Nicoly,
Jonathan I. Segal,
Jonathan St. George,
Soyon Kim,
Kevin Ching,
Francisco R. Ortega,
Hee Rin Lee
Abstract:
How might healthcare workers (HCWs) leverage augmented reality head-mounted displays (AR-HMDs) to enhance teamwork? Although AR-HMDs have shown immense promise in supporting teamwork in healthcare settings, design for Emergency Department (ER) teams has received little attention. The ER presents unique challenges, including procedural recall, medical errors, and communication gaps. To address this…
▽ More
How might healthcare workers (HCWs) leverage augmented reality head-mounted displays (AR-HMDs) to enhance teamwork? Although AR-HMDs have shown immense promise in supporting teamwork in healthcare settings, design for Emergency Department (ER) teams has received little attention. The ER presents unique challenges, including procedural recall, medical errors, and communication gaps. To address this gap, we engaged in a participatory design study with healthcare workers to gain a deep understanding of the potential for AR-HMDs to facilitate teamwork during ER procedures. Our results reveal that AR-HMDs can be used as an information-sharing and information-retrieval system to bridge knowledge gaps, and concerns about integrating AR-HMDs in ER workflows. We contribute design recommendations for seven role-based AR-HMD application scenarios involving HCWs with various expertise, working across multiple medical tasks. We hope our research inspires designers to embark on the development of new AR-HMD applications for high-stakes, team environments.
△ Less
Submitted 24 February, 2025;
originally announced February 2025.
-
Enhancing Well-Being Through Positive Technology: VR Forest Bathing
Authors:
Francisco R. Ortega,
Victoria Interrante,
Sara Lotemplio,
Rachel Masters,
Jalynn Nicoly,
Zahra Borhani,
Deana Davalos,
Daniel Zielasko
Abstract:
The growing demand for accessible therapeutic options has led to the exploration of Virtual Reality (VR) as a platform for forest bathing, which aims to reduce stress and improve cognitive functions. This paper brings together findings from three studies by the authors. The first study compared environments with and without plant life to examine how biomass influences stress reduction. The second…
▽ More
The growing demand for accessible therapeutic options has led to the exploration of Virtual Reality (VR) as a platform for forest bathing, which aims to reduce stress and improve cognitive functions. This paper brings together findings from three studies by the authors. The first study compared environments with and without plant life to examine how biomass influences stress reduction. The second study focused on the differences between low-fidelity and high-fidelity VR environments, while the third explored whether the benefits of VR forest bathing come from being immersed in realistic environments or simply from viewing something beautiful. The results showed no significant differences between environments with and without biomass, but highlighted the positive effects of high-fidelity VR environments and realistic nature over abstract art. The paper also covers how VR nature experiences may boost executive functioning and well-being in older adults and discusses the potential of generative AI to create customized VR environments. It concludes with a call for further collaborative research to refine VR forest bathing for stress relief and cognitive enhancement.
△ Less
Submitted 9 November, 2024;
originally announced November 2024.
-
Ping! Your Food is Ready: Comparing Different Notification Techniques in 3D AR Cooking Environment
Authors:
Aditya Raikwar,
Lucas Plabst,
Anil Ufuk Batmaz,
Florian Niebling,
Francisco R. Ortega
Abstract:
Implementing visual and audio notifications on augmented reality devices is a crucial element of intuitive and easy-to-use interfaces. In this paper, we explored creating intuitive interfaces through visual and audio notifications. The study evaluated user performance and preference across three conditions: visual notifications in fixed positions, visual notifications above objects, and no visual…
▽ More
Implementing visual and audio notifications on augmented reality devices is a crucial element of intuitive and easy-to-use interfaces. In this paper, we explored creating intuitive interfaces through visual and audio notifications. The study evaluated user performance and preference across three conditions: visual notifications in fixed positions, visual notifications above objects, and no visual notifications with monaural sounds. The users were tasked with cooking and serving customers in an open-source Augmented-Reality sandbox environment called ARtisan Bistro. The results indicated that visual notifications above objects combined with localized audio feedback were the most effective and preferred method by participants. The findings highlight the importance of strategic placement of visual and audio notifications in AR, providing insights for engineers and developers to design intuitive 3D user interfaces.
△ Less
Submitted 17 September, 2024;
originally announced September 2024.
-
Multimedia and Immersive Training Materials Influence Impressions of Learning But Not Learning Outcomes
Authors:
Benjamin A. Clegg,
Alex Karduna,
Ethan Holen,
Jason Garcia,
Matthew G. Rhodes,
Francisco R. Ortega
Abstract:
Although the use of technologies like multimedia and virtual reality (VR) in training offer the promise of improved learning, these richer and potentially more engaging materials do not consistently produce superior learning outcomes. Default approaches to such training may inadvertently mimic concepts like naive realism in display design, and desirable difficulties in the science of learning - fo…
▽ More
Although the use of technologies like multimedia and virtual reality (VR) in training offer the promise of improved learning, these richer and potentially more engaging materials do not consistently produce superior learning outcomes. Default approaches to such training may inadvertently mimic concepts like naive realism in display design, and desirable difficulties in the science of learning - fostering an impression of greater learning dissociated from actual gains in memory. This research examined the influence of format of instructions in learning to assemble items from components. Participants in two experiments were trained on the steps to assemble a series of bars, that resembled Meccano pieces, into eight different shapes. After training on pairs of shapes, participants rated the likelihood they would remember the shapes and then were administered a recognition test. Relative to viewing a static diagram, viewing videos of shapes being constructed in a VR environment (Experiment 1) or viewing within an immersive VR system (Experiment 2) elevated participants' assessments of their learning but without enhancing learning outcomes. Overall, these findings illustrate how future workers might mistakenly come to believe that technologically advanced support improves learning and prefer instructional designs that integrate similarly complex cues into training.
△ Less
Submitted 7 July, 2024;
originally announced July 2024.
-
Fast, Accurate, but Sometimes Too-Compelling Support: The Impact of Imperfectly Automated Cues in an Augmented Reality Head-Mounted Display on Visual Search Performance
Authors:
Amelia C. Warden,
Christopher D. Wickens,
Daniel Rehberg,
Francisco R. Ortega,
Benjamin A. Clegg
Abstract:
While visual search for targets within a complex scene might benefit from using augmented-reality (AR) head-mounted display (HMD) technologies helping to efficiently direct human attention, imperfectly reliable automation support could manifest in occasional errors. The current study examined the effectiveness of different HMD cues that might support visual search performance and their respective…
▽ More
While visual search for targets within a complex scene might benefit from using augmented-reality (AR) head-mounted display (HMD) technologies helping to efficiently direct human attention, imperfectly reliable automation support could manifest in occasional errors. The current study examined the effectiveness of different HMD cues that might support visual search performance and their respective consequences following automation errors. Fifty-six participants searched a 3D environment containing 48 objects in a room, in order to locate a target object that was viewed prior to each trial. They searched either unaided or assisted by one of three HMD types of cues: an arrow pointing to the target, a plan-view minimap highlighting the target, and a constantly visible icon depicting the appearance of the target object. The cue was incorrect on 17% of the trials for one group of participants and 100% correct for the second group. Through both analysis and modeling of both search speed and accuracy, the results indicated that the arrow and minimap cues depicting location information were more effective than the icon cue depicting visual appearance, both overall, and when the cue was correct. However, there was a tradeoff on the infrequent occasions when the cue erred. The most effective AR-based cue led to a greater automation bias, in which the cue was more often blindly followed without careful examination of the raw images. The results speak to the benefits of augmented reality and the need to examine potential costs when AR-conveyed information may be incorrect because of imperfectly reliable systems.
△ Less
Submitted 24 March, 2023;
originally announced March 2023.
-
Eliciting Multimodal Gesture+Speech Interactions in a Multi-Object Augmented Reality Environment
Authors:
Xiaoyan Zhou,
Adam S. Williams,
Francisco R. Ortega
Abstract:
As augmented reality technology and hardware become more mature and affordable, researchers have been exploring more intuitive and discoverable interaction techniques for immersive environments. In this paper, we investigate multimodal interaction for 3D object manipulation in a multi-object virtual environment. To identify the user-defined gestures, we conducted an elicitation study involving 24…
▽ More
As augmented reality technology and hardware become more mature and affordable, researchers have been exploring more intuitive and discoverable interaction techniques for immersive environments. In this paper, we investigate multimodal interaction for 3D object manipulation in a multi-object virtual environment. To identify the user-defined gestures, we conducted an elicitation study involving 24 participants for 22 referents with an augmented reality headset. It yielded 528 proposals and generated a winning gesture set with 25 gestures after binning and ranking all gesture proposals. We found that for the same task, the same gesture was preferred for both one and two object manipulation, although both hands were used in the two object scenario. We presented the gestures and speech results, and the differences compared to similar studies in a single object virtual environment. The study also explored the association between speech expressions and gesture stroke during object manipulation, which could improve the recognizer efficiency in augmented reality headsets.
△ Less
Submitted 25 July, 2022;
originally announced July 2022.
-
A Pilot Study on The Impact of Stereoscopic Display Type on User Interactions Within A Immersive Analytics Environment
Authors:
Adam S. Williams,
Xiaoyan Zhou,
Michel Pahud,
Francisco R. Ortega
Abstract:
Immersive Analytics (IA) and consumer adoption of augmented reality (AR) and virtual reality (VR) head-mounted displays (HMDs) are both rapidly growing. When used in conjunction, stereoscopic IA environments can offer improved user understanding and engagement; however, it is unclear how the choice of stereoscopic display impacts user interactions within an IA environment. This paper presents a pi…
▽ More
Immersive Analytics (IA) and consumer adoption of augmented reality (AR) and virtual reality (VR) head-mounted displays (HMDs) are both rapidly growing. When used in conjunction, stereoscopic IA environments can offer improved user understanding and engagement; however, it is unclear how the choice of stereoscopic display impacts user interactions within an IA environment. This paper presents a pilot study that examines the impact of stereoscopic display type on object manipulation and environmental navigation using consumer-available AR and VR displays. This work finds that the display type can impact how users manipulate virtual content, how they navigate the environment, and how able they are to answer questions about the represented data.
△ Less
Submitted 25 July, 2022;
originally announced July 2022.
-
CLEDGE: A Hybrid Cloud-Edge Computing Framework over Information Centric Networking
Authors:
Md Washik Al Azad,
Susmit Shannigrahi,
Nicholas Stergiou,
Francisco R. Ortega,
Spyridon Mastorakis
Abstract:
In today's era of Internet of Things (IoT), where massive amounts of data are produced by IoT and other devices, edge computing has emerged as a prominent paradigm for low-latency data processing. However, applications may have diverse latency requirements: certain latency-sensitive processing operations may need to be performed at the edge, while delay-tolerant operations can be performed on the…
▽ More
In today's era of Internet of Things (IoT), where massive amounts of data are produced by IoT and other devices, edge computing has emerged as a prominent paradigm for low-latency data processing. However, applications may have diverse latency requirements: certain latency-sensitive processing operations may need to be performed at the edge, while delay-tolerant operations can be performed on the cloud, without occupying the potentially limited edge computing resources. To achieve that, we envision an environment where computing resources are distributed across edge and cloud offerings. In this paper, we present the design of CLEDGE (CLoud + EDGE), an information-centric hybrid cloud-edge framework, aiming to maximize the on-time completion of computational tasks offloaded by applications with diverse latency requirements. The design of CLEDGE is motivated by the networking challenges that mixed reality researchers face. Our evaluation demonstrates that CLEDGE can complete on-time more than 90% of offloaded tasks with modest overheads.
△ Less
Submitted 15 July, 2021;
originally announced July 2021.
-
A Concise Guide to Elicitation Methodology
Authors:
Adam S. Williams,
Francisco R. Ortega
Abstract:
One of the open questions in the field of interaction design is "what inputs or interaction techniques should be used with augmented reality devices?" The transition from a touchpad and a keyboard to a multi-touch device was relatively small. The transition from a multi-touch device to an HMD with no controllers or clear surface to interact with is more complicated. This book is a guide for how to…
▽ More
One of the open questions in the field of interaction design is "what inputs or interaction techniques should be used with augmented reality devices?" The transition from a touchpad and a keyboard to a multi-touch device was relatively small. The transition from a multi-touch device to an HMD with no controllers or clear surface to interact with is more complicated. This book is a guide for how to figure out what interaction techniques and modalities people prefer when interacting with those devices. The name of the technique covered here is Elicitation. Elicitation is a form of participatory design, meaning design with direct end-user involvement. By running an elicitation study researchers can observe unconstrained human interactions with emerging technologies to help guide input design.
△ Less
Submitted 26 May, 2021;
originally announced May 2021.
-
Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation
Authors:
Adam S. Williams,
Francisco R. Ortega
Abstract:
This research establishes a better understanding of the syntax choices in speech interactions and of how speech, gesture, and multimodal gesture and speech interactions are produced by users in unconstrained object manipulation environments using augmented reality. The work presents a multimodal elicitation study conducted with 24 participants. The canonical referents for translation, rotation, an…
▽ More
This research establishes a better understanding of the syntax choices in speech interactions and of how speech, gesture, and multimodal gesture and speech interactions are produced by users in unconstrained object manipulation environments using augmented reality. The work presents a multimodal elicitation study conducted with 24 participants. The canonical referents for translation, rotation, and scale were used along with some abstract referents (create, destroy, and select). In this study time windows for gesture and speech multimodal interactions are developed using the start and stop times of gestures and speech as well as the stoke times for gestures. While gestures commonly precede speech by 81 ms we find that the stroke of the gesture is commonly within 10 ms of the start of speech. Indicating that the information content of a gesture and its co-occurring speech are well aligned to each other. Lastly, the trends across the most common proposals for each modality are examined. Showing that the disagreement between proposals is often caused by a variation of hand posture or syntax. Allowing us to present aliasing recommendations to increase the percentage of users' natural interactions captured by future multimodal interactive systems.
△ Less
Submitted 8 August, 2022; v1 submitted 14 September, 2020;
originally announced September 2020.
-
Towards a Virtual Reality Home IoT Network Visualizer
Authors:
Drew Johnston,
Jarret Flack,
Indrakshi Ray,
Francisco R. Ortega
Abstract:
We present an IoT home network visualizer that utilizes virtual reality (VR). This prototype demonstrates the potential that VR has to aid in the understanding of home IoT networks. This is particularly important due the increased number of household devices now connected to the Internet. This prototype is able to function in a standard display or a VR headset. A prototype was developed to aid in…
▽ More
We present an IoT home network visualizer that utilizes virtual reality (VR). This prototype demonstrates the potential that VR has to aid in the understanding of home IoT networks. This is particularly important due the increased number of household devices now connected to the Internet. This prototype is able to function in a standard display or a VR headset. A prototype was developed to aid in the understanding of home IoT networks for homeowners.
△ Less
Submitted 17 January, 2020;
originally announced January 2020.
-
City Planning with Augmented Reality
Authors:
Catherine Angelini,
Adam S. Williams,
Mathew Kress,
Edgar Ramos Vieira,
Newton D'Souza,
Naphtali D. Rishe,
Joseph Medina,
Francisco R. Ortega
Abstract:
We present an early study designed to analyze how city planning and the health of senior citizens can benefit from the use of augmented reality (AR) using Microsoft's HoloLens. We also explore whether AR and VR can be used to help city planners receive real-time feedback from citizens, such as the elderly, on virtual plans, allowing for informed decisions to be made before any construction begins.
We present an early study designed to analyze how city planning and the health of senior citizens can benefit from the use of augmented reality (AR) using Microsoft's HoloLens. We also explore whether AR and VR can be used to help city planners receive real-time feedback from citizens, such as the elderly, on virtual plans, allowing for informed decisions to be made before any construction begins.
△ Less
Submitted 17 January, 2020;
originally announced January 2020.
-
The NGC7771+NGC7770 Minor Merger: Harassing the Little One?
Authors:
Almudena Alonso-Herrero,
F. Fabian Rosales Ortega,
Sebastian F. Sanchez,
Robert C. Kennicutt,
Miguel Pereira-Santaella,
Angeles I. Diaz
Abstract:
Numerical simulations of minor mergers, typically having mass ratios greater than 3:1, predict little enhancement in the global star formation activity. However, these models also predict that the satellite galaxy is more susceptible to the effects of the interaction than the primary. We use optical integral field spectroscopy and deep optical imaging to study the NGC7771+NGC7770 interacting syste…
▽ More
Numerical simulations of minor mergers, typically having mass ratios greater than 3:1, predict little enhancement in the global star formation activity. However, these models also predict that the satellite galaxy is more susceptible to the effects of the interaction than the primary. We use optical integral field spectroscopy and deep optical imaging to study the NGC7771+NGC7770 interacting system (~10:1 stellar mass ratio) to test these predictions. We find that the satellite galaxy NGC7770 is currently experiencing a galaxy-wide starburst with most of the optical light being from young and post-starburst stellar populations(<1Gyr). This galaxy lies off of the local star-forming sequence for composite galaxies with an enhanced integrated specific star formation rate. We also detect in the outskirts of NGC7770 Halpha emitting gas filaments. This gas appears to have been stripped from one of the two galaxies and is being excited by shocks. All these results are consistent with a minor-merger induced episode(s) of star formation in NGC7770 after the first close passage. Such effects are not observed on the primary galaxy NGC7771.
△ Less
Submitted 8 June, 2012;
originally announced June 2012.
