Skip to main content

Showing 1–20 of 20 results for author: Froehlich, J E

Searching in archive cs. Search in all archives.
.
  1. arXiv:2504.21360  [pdf, other

    cs.HC

    ImaginateAR: AI-Assisted In-Situ Authoring in Augmented Reality

    Authors: Jaewook Lee, Filippo Aleotti, Diego Mazala, Guillermo Garcia-Hernando, Sara Vicente, Oliver James Johnston, Isabel Kraus-Liang, Jakub Powierza, Donghoon Shin, Jon E. Froehlich, Gabriel Brostow, Jessica Van Brummelen

    Abstract: While augmented reality (AR) enables new ways to play, tell stories, and explore ideas rooted in the physical world, authoring personalized AR content remains difficult for non-experts, often requiring professional tools and time. Prior systems have explored AI-driven XR design but typically rely on manually-defined environments and fixed asset libraries, limiting creative flexibility and real-wor… ▽ More

    Submitted 30 April, 2025; originally announced April 2025.

  2. Accessibility for Whom? Perceptions of Sidewalk Barriers Across Disability Groups and Implications for Designing Personalized Maps

    Authors: Chu Li, Rock Yuren Pang, Delphine Labbé, Yochai Eisenberg, Maryam Hosseini, Jon E. Froehlich

    Abstract: Despite diverse mobility needs worldwide, existing mapping tools fail to address the varied experiences of different mobility device users. This paper presents a large-scale online survey exploring how five mobility groups -- users of canes, walkers, mobility scooters, manual wheelchairs, and motorized wheelchairs -- perceive sidewalk barriers. Using 52 sidewalk barrier images, respondents evaluat… ▽ More

    Submitted 27 February, 2025; originally announced February 2025.

    Comments: Manuscript accepted at CHI'25

  3. ArtInsight: Enabling AI-Powered Artwork Engagement for Mixed Visual-Ability Families

    Authors: Arnavi Chheda-Kothary, Ritesh Kanchi, Chris Sanders, Kevin Xiao, Aditya Sengupta, Melanie Kneitmix, Jacob O. Wobbrock, Jon E. Froehlich

    Abstract: We introduce ArtInsight, a novel AI-powered system to facilitate deeper engagement with child-created artwork in mixed visual-ability families. ArtInsight leverages large language models (LLMs) to craft a respectful and thorough initial description of a child's artwork, and provides: creative AI-generated descriptions for a vivid overview, audio recording to capture the child's own description of… ▽ More

    Submitted 10 March, 2025; v1 submitted 26 February, 2025; originally announced February 2025.

    Comments: 21 pages, 30th International Conference on Intelligent User Interfaces (IUI 2025)

    Journal ref: 30th International Conference on Intelligent User Interfaces 2025

  4. arXiv:2410.20571  [pdf, other

    cs.HC

    Making Urban Art Accessible: Current Art Access Techniques, Design Considerations, and the Role of AI

    Authors: Lucy Jiang, Jon E. Froehlich, Leah Findlater

    Abstract: Public artwork, from vibrant wall murals to captivating sculptures, can enhance the aesthetic of urban spaces, foster a sense of community and cultural identity, and help attract visitors. Despite its benefits, most public art is visual, making it often inaccessible to blind and low vision (BLV) people. In this workshop paper, we first draw on art literature to help define the space of public art,… ▽ More

    Submitted 27 October, 2024; originally announced October 2024.

    Comments: ASSETS 2024 Workshop Submission (The Future of Urban Accessibility: The Role of AI)

  5. Engaging with Children's Artwork in Mixed Visual-Ability Families

    Authors: Arnavi Chheda-Kothary, Jacob O. Wobbrock, Jon E. Froehlich

    Abstract: We present two studies exploring how blind or low-vision (BLV) family members engage with their sighted children's artwork, strategies to support understanding and interpretation, and the potential role of technology, such as AI, therein. Our first study involved 14 BLV individuals, and the second included five groups of BLV individuals with their children. Through semi-structured interviews with… ▽ More

    Submitted 30 July, 2024; v1 submitted 26 July, 2024; originally announced July 2024.

  6. arXiv:2407.13515  [pdf, other

    cs.HC

    CookAR: Affordance Augmentations in Wearable AR to Support Kitchen Tool Interactions for People with Low Vision

    Authors: Jaewook Lee, Andrew D. Tjahjadi, Jiho Kim, Junpu Yu, Minji Park, Jiawen Zhang, Jon E. Froehlich, Yapeng Tian, Yuhang Zhao

    Abstract: Cooking is a central activity of daily living, supporting independence as well as mental and physical health. However, prior work has highlighted key barriers for people with low vision (LV) to cook, particularly around safely interacting with tools, such as sharp knives or hot pans. Drawing on recent advancements in computer vision (CV), we present CookAR, a head-mounted AR system with real-time… ▽ More

    Submitted 27 July, 2024; v1 submitted 18 July, 2024; originally announced July 2024.

  7. AltGeoViz: Facilitating Accessible Geovisualization

    Authors: Chu Li, Rock Yuren Pang, Ather Sharif, Arnavi Chheda-Kothary, Jeffrey Heer, Jon E. Froehlich

    Abstract: Geovisualizations are powerful tools for exploratory spatial analysis, enabling sighted users to discern patterns, trends, and relationships within geographic data. However, these visual tools have remained largely inaccessible to screen-reader users. We present AltGeoViz, a new system we designed to facilitate geovisualization exploration for these users. AltGeoViz dynamically generates alt-text… ▽ More

    Submitted 9 December, 2024; v1 submitted 19 June, 2024; originally announced June 2024.

    Comments: IEEE VIS 2024

  8. arXiv:2405.07089  [pdf, other

    cs.HC

    SonifyAR: Context-Aware Sound Generation in Augmented Reality

    Authors: Xia Su, Jon E. Froehlich, Eunyee Koh, Chang Xiao

    Abstract: Sound plays a crucial role in enhancing user experience and immersiveness in Augmented Reality (AR). However, current platforms lack support for AR sound authoring due to limited interaction types, challenges in collecting and specifying context information, and difficulty in acquiring matching sound assets. We present SonifyAR, an LLM-based AR sound authoring system that generates context-aware s… ▽ More

    Submitted 11 August, 2024; v1 submitted 11 May, 2024; originally announced May 2024.

    Comments: To appear in UIST2024

  9. arXiv:2404.08213  [pdf, other

    cs.HC

    GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Reality

    Authors: Jaewook Lee, Jun Wang, Elizabeth Brown, Liam Chu, Sebastian S. Rodriguez, Jon E. Froehlich

    Abstract: Voice assistants (VAs) like Siri and Alexa are transforming human-computer interaction; however, they lack awareness of users' spatiotemporal context, resulting in limited performance and unnatural dialogue. We introduce GazePointAR, a fully-functional context-aware VA for wearable augmented reality that leverages eye gaze, pointing gestures, and conversation history to disambiguate speech queries… ▽ More

    Submitted 11 April, 2024; originally announced April 2024.

  10. arXiv:2403.09810  [pdf, other

    cs.HC cs.AI cs.LG

    LabelAId: Just-in-time AI Interventions for Improving Human Labeling Quality and Domain Knowledge in Crowdsourcing Systems

    Authors: Chu Li, Zhihan Zhang, Michael Saugstad, Esteban Safranchik, Minchu Kulkarni, Xiaoyu Huang, Shwetak Patel, Vikram Iyer, Tim Althoff, Jon E. Froehlich

    Abstract: Crowdsourcing platforms have transformed distributed problem-solving, yet quality control remains a persistent challenge. Traditional quality control measures, such as prescreening workers and refining instructions, often focus solely on optimizing economic output. This paper explores just-in-time AI interventions to enhance both labeling quality and domain-specific knowledge among crowdworkers. W… ▽ More

    Submitted 14 March, 2024; originally announced March 2024.

  11. arXiv:2311.12225  [pdf

    cs.CV

    HandSight: DeCAF & Improved Fisher Vectors to Classify Clothing Color and Texture with a Finger-Mounted Camera

    Authors: Alexander J. Medeiros, Lee Stearns, Jon E. Froehlich

    Abstract: We demonstrate the use of DeCAF and Improved Fisher Vector image features to classify clothing texture. The issue of choosing clothes is a problem for the blind every day. This work attempts to solve the issue with a finger-mounted camera and state-of-the-art classification algorithms. To evaluate our solution, we collected 520 close-up images across 29 pieces of clothing. We contribute (1) the HC… ▽ More

    Submitted 20 November, 2023; originally announced November 2023.

    Comments: 10 pages, 15 figures

    MSC Class: H.5.m

  12. arXiv:2211.11545  [pdf

    physics.soc-ph cs.CY

    A Pilot Study of Sidewalk Equity in Seattle Using Crowdsourced Sidewalk Assessment Data

    Authors: Chu Li, Lisa Orii, Mikey Saugstad, Stephen J. Mooney, Yochai Eisenberg, Delphine Labbé, Joy Hammel, Jon E. Froehlich

    Abstract: We examine the potential of using large-scale open crowdsourced sidewalk data from Project Sidewalk to study the distribution and condition of sidewalks in Seattle, WA. While potentially noisier than professionally gathered sidewalk datasets, crowdsourced data enables large, cross-regional studies that would be otherwise expensive and difficult to manage. As an initial case study, we examine spati… ▽ More

    Submitted 5 October, 2022; originally announced November 2022.

    Comments: Workshop paper presented at "The 1st ASSETS'22 Workshop on The Future or urban Accessibility (UrbanAccess'22)"

  13. arXiv:2210.02533  [pdf

    cs.HC cs.CV cs.CY

    Towards Semi-automatic Detection and Localization of Indoor Accessibility Issues using Mobile Depth Scanning and Computer Vision

    Authors: Xia Su, Kaiming Cheng, Han Zhang, Jaewook Lee, Jon E. Froehlich

    Abstract: To help improve the safety and accessibility of indoor spaces, researchers and health professionals have created assessment instruments that enable homeowners and trained experts to audit and improve homes. With advances in computer vision, augmented reality (AR), and mobile sensors, new approaches are now possible. We introduce RASSAR (Room Accessibility and Safety Scanning in Augmented Reality),… ▽ More

    Submitted 5 October, 2022; originally announced October 2022.

    Comments: Workshop paper presented at "The 1st ASSETS'22 Workshop on The Future or urban Accessibility (UrbanAccess'22)"

  14. arXiv:2207.13626  [pdf

    cs.HC cs.CY

    Towards Mapping and Assessing Sidewalk Accessibility Across Sociocultural and Geographic Contexts

    Authors: Jon E. Froehlich, Michael Saugstad, Manaswi Saha, Matthew Johnson

    Abstract: Despite the important role of sidewalks in supporting mobility, accessibility, and public health, there is a lack of high-quality datasets and corresponding analyses on sidewalk existence and condition. Our work explores a twofold vision: first, to develop scalable mechanisms to locate and assess sidewalks in cities across the world, and second, to use this data to support new urban analyses and m… ▽ More

    Submitted 27 July, 2022; originally announced July 2022.

    Comments: Workshop paper at the Data4Good workshop hosted as part of AVI2020. See https://sites.google.com/view/data4good/program

  15. arXiv:2206.13677  [pdf, other

    cs.CV cs.HC

    Towards Global-Scale Crowd+AI Techniques to Map and Assess Sidewalks for People with Disabilities

    Authors: Maryam Hosseini, Mikey Saugstad, Fabio Miranda, Andres Sevtsuk, Claudio T. Silva, Jon E. Froehlich

    Abstract: There is a lack of data on the location, condition, and accessibility of sidewalks across the world, which not only impacts where and how people travel but also fundamentally limits interactive mapping tools and urban analytics. In this paper, we describe initial work in semi-automatically building a sidewalk network topology from satellite imagery using hierarchical multi-scale attention models,… ▽ More

    Submitted 18 August, 2022; v1 submitted 27 June, 2022; originally announced June 2022.

    Comments: CVPR 2022 AVA (Accessibility, Vision, and Autonomy Meet) Workshop

  16. arXiv:2202.11134  [pdf

    cs.HC cs.LG cs.SD eess.AS

    ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users

    Authors: Dhruv Jain, Khoa Huynh Anh Nguyen, Steven Goodman, Rachel Grossman-Kahn, Hung Ngo, Aditya Kusupati, Ruofei Du, Alex Olwal, Leah Findlater, Jon E. Froehlich

    Abstract: Recent advances have enabled automatic sound recognition systems for deaf and hard of hearing (DHH) users on mobile devices. However, these tools use pre-trained, generic sound recognition models, which do not meet the diverse needs of DHH users. We introduce ProtoSound, an interactive system for customizing sound recognition models by recording a few examples, thereby enabling personalized and fi… ▽ More

    Submitted 22 February, 2022; originally announced February 2022.

    Comments: Published at the ACM CHI Conference on Human Factors in Computing Systems (CHI) 2022

  17. Social, Environmental, and Technical: Factors at Play in the Current Use and Future Design of Small-Group Captioning

    Authors: Emma J. McDonnell, Ping Liu, Steven M. Goodman, Raja Kushalnagar, Jon E. Froehlich, Leah Findlater

    Abstract: Real-time captioning is a critical accessibility tool for many d/Deaf and hard of hearing (DHH) people. While the vast majority of captioning work has focused on formal settings and technical innovations, in contrast, we investigate captioning for informal, interactive small-group conversations, which have a high degree of spontaneity and foster dynamic social interactions. This paper reports on s… ▽ More

    Submitted 21 September, 2021; originally announced September 2021.

    Comments: 25 pages, 3 figures, to be published in the PACMHCI-CSCW2 October 2021 edition, to be presented at CSCW 2021

  18. A bibliometric analysis of citation diversity in accessibility and HCI research

    Authors: Lucy Lu Wang, Kelly Mack, Emma McDonnell, Dhruv Jain, Leah Findlater, Jon E. Froehlich

    Abstract: Accessibility research sits at the junction of several disciplines, drawing influence from HCI, disability studies, psychology, education, and more. To characterize the influences and extensions of accessibility research, we undertake a study of citation trends for accessibility and related HCI communities. We assess the diversity of venues and fields of study represented among the referenced and… ▽ More

    Submitted 11 March, 2021; originally announced March 2021.

    Comments: 11 pages, 5 figures, 3 tables, 2 appendices; CHI LBW 2021; accessible PDF available at https://makeabilitylab.cs.washington.edu/

  19. What Do We Mean by "Accessibility Research"? A Literature Survey of Accessibility Papers in CHI and ASSETS from 1994 to 2019

    Authors: Kelly Mack, Emma McDonnell, Dhruv Jain, Lucy Lu Wang, Jon E. Froehlich, Leah Findlater

    Abstract: Accessibility research has grown substantially in the past few decades, yet there has been no literature review of the field. To understand current and historical trends, we created and analyzed a dataset of accessibility papers appearing at CHI and ASSETS since ASSETS' founding in 1994. We qualitatively coded areas of focus and methodological decisions for the past 10 years (2010-2019, N=506 pape… ▽ More

    Submitted 3 February, 2021; v1 submitted 11 January, 2021; originally announced January 2021.

  20. arXiv:1912.07456  [pdf

    cs.CY cs.HC

    Organizing Family Support Services at ACM Conferences

    Authors: Audrey Girouard, Jon E. Froehlich, Regan Mandryk, Mark Hancock

    Abstract: This article reflects on our experiences providing family-support services to a large, interdisciplinary ACM conference (CHI2018) including, the policy decisions, the challenges, and the successes. The article incorporates empirical data collected from pre- and post-conference surveys, observed use of the services, and aspirational aims for future conferences. We are discussing best practices and… ▽ More

    Submitted 12 December, 2019; originally announced December 2019.

    Comments: This article is a complement to an abbreviated version published in the April 2020 issue of the Communications of the ACM. This longer version includes significantly more information on our approach to planning family services at CHI2018, decision making, and our pre-conference and post-conference assessments