-
What Does Success Look Like? Catalyzing Meeting Intentionality with AI-Assisted Prospective Reflection
Authors:
Ava Elizabeth Scott,
Lev Tankelevitch,
Payod Panda,
Rishi Vanukuru,
Xinyue Chen,
Sean Rintel
Abstract:
Despite decades of HCI and Meeting Science research, complaints about ineffective meetings are still pervasive. We argue that meeting technologies lack support for prospective reflection, that is, thinking about why a meeting is needed and what might happen. To explore this, we designed a Meeting Purpose Assistant (MPA) technology probe to coach users to articulate their meeting's purpose and chal…
▽ More
Despite decades of HCI and Meeting Science research, complaints about ineffective meetings are still pervasive. We argue that meeting technologies lack support for prospective reflection, that is, thinking about why a meeting is needed and what might happen. To explore this, we designed a Meeting Purpose Assistant (MPA) technology probe to coach users to articulate their meeting's purpose and challenges, and act accordingly. The MPA used Generative AI to support personalized and actionable prospective reflection across the diversity of meeting contexts. Using a participatory prompting methodology, 18 employees of a global technology company reflected with the MPA on upcoming meetings. Observed impacts were: clarifying meeting purposes, challenges, and success conditions; changing perspectives and flexibility; improving preparation and communication; and proposing changed plans. We also identify perceived social, temporal, and technological barriers to using the MPA. We present system and workflow design considerations for developing AI-assisted reflection support for meetings.
△ Less
Submitted 20 May, 2025;
originally announced May 2025.
-
Are We On Track? AI-Assisted Active and Passive Goal Reflection During Meetings
Authors:
Xinyue Chen,
Lev Tankelevitch,
Rishi Vanukuru,
Ava Elizabeth Scott,
Payod Panda,
Sean Rintel
Abstract:
Meetings often suffer from a lack of intentionality, such as unclear goals and straying off-topic. Identifying goals and maintaining their clarity throughout a meeting is challenging, as discussions and uncertainties evolve. Yet meeting technologies predominantly fail to support meeting intentionality. AI-assisted reflection is a promising approach. To explore this, we conducted a technology probe…
▽ More
Meetings often suffer from a lack of intentionality, such as unclear goals and straying off-topic. Identifying goals and maintaining their clarity throughout a meeting is challenging, as discussions and uncertainties evolve. Yet meeting technologies predominantly fail to support meeting intentionality. AI-assisted reflection is a promising approach. To explore this, we conducted a technology probe study with 15 knowledge workers, integrating their real meeting data into two AI-assisted reflection probes: a passive and active design. Participants identified goal clarification as a foundational aspect of reflection. Goal clarity enabled people to assess when their meetings were off-track and reprioritize accordingly. Passive AI intervention helped participants maintain focus through non-intrusive feedback, while active AI intervention, though effective at triggering immediate reflection and action, risked disrupting the conversation flow. We identify three key design dimensions for AI-assisted reflection systems, and provide insights into design trade-offs, emphasizing the need to adapt intervention intensity and timing, balance democratic input with efficiency, and offer user control to foster intentional, goal-oriented behavior during meetings and beyond.
△ Less
Submitted 7 April, 2025; v1 submitted 1 April, 2025;
originally announced April 2025.
-
The New Calculator? Practices, Norms, and Implications of Generative AI in Higher Education
Authors:
Auste Simkute,
Viktor Kewenig,
Abigail Sellen,
Sean Rintel,
Lev Tankelevitch
Abstract:
Generative AI (GenAI) has introduced myriad opportunities and challenges for higher education. Anticipating this potential transformation requires understanding students' contextualised practices and norms around GenAI. We conducted semi-structured interviews with 26 students and 11 educators from diverse departments across two universities. Grounded in Strong Structuration Theory, we find diversi…
▽ More
Generative AI (GenAI) has introduced myriad opportunities and challenges for higher education. Anticipating this potential transformation requires understanding students' contextualised practices and norms around GenAI. We conducted semi-structured interviews with 26 students and 11 educators from diverse departments across two universities. Grounded in Strong Structuration Theory, we find diversity in students' uses and motivations for GenAI. Occurring in the context of unclear university guidelines, institutional fixation on plagiarism, and inconsistent educator communication, students' practices are informed by unspoken rules around appropriate use, GenAI limitations and reliance strategies, and consideration of agency and skills. Perceived impacts include changes in confidence, and concerns about skill development, relationships with educators, and plagiarism. Both groups envision changes in universities' attitude to GenAI, responsible use training, assessments, and integration of GenAI into education. We discuss socio-technical implications in terms of current and anticipated changes in the external and internal structures that contextualise students' GenAI use.
△ Less
Submitted 15 January, 2025;
originally announced January 2025.
-
Nods of Agreement: Webcam-Driven Avatars Improve Meeting Outcomes and Avatar Satisfaction Over Audio-Driven or Static Avatars in All-Avatar Work Videoconferencing
Authors:
Fang Ma,
Ju Zhang,
Lev Tankelevitch,
Payod Panda,
Torang Asadi,
Charlie Hewitt,
Lohit Petikam,
James Clemoes,
Marco Gillies,
Xueni Pan,
Sean Rintel,
Marta Wilczkowiak
Abstract:
Avatars are edging into mainstream videoconferencing, but evaluation of how avatar animation modalities contribute to work meeting outcomes has been limited. We report a within-group videoconferencing experiment in which 68 employees of a global technology company, in 16 groups, used the same stylized avatars in three modalities (static picture, audio-animation, and webcam-animation) to complete c…
▽ More
Avatars are edging into mainstream videoconferencing, but evaluation of how avatar animation modalities contribute to work meeting outcomes has been limited. We report a within-group videoconferencing experiment in which 68 employees of a global technology company, in 16 groups, used the same stylized avatars in three modalities (static picture, audio-animation, and webcam-animation) to complete collaborative decision-making tasks. Quantitatively, for meeting outcomes, webcam-animated avatars improved meeting effectiveness over the picture modality and were also reported to be more comfortable and inclusive than both other modalities. In terms of avatar satisfaction, there was a similar preference for webcam animation as compared to both other modalities. Our qualitative analysis shows participants expressing a preference for the holistic motion of webcam animation, and that meaningful movement outweighs realism for meeting outcomes, as evidenced through a systematic overview of ten thematic factors. We discuss implications for research and commercial deployment and conclude that webcam-animated avatars are a plausible alternative to video in work meetings.
△ Less
Submitted 17 January, 2025; v1 submitted 17 December, 2024;
originally announced December 2024.
-
The Future of Skill: What Is It to Be Skilled at Work?
Authors:
Axel Niklasson,
Sean Rintel,
Stephann Makri,
Alex Taylor
Abstract:
In this short paper, we introduce work that is aiming to purposefully venture into this mesh of questions from a different starting point. Interjecting into the conversation, we want to ask: 'What is it to be skilled at work?' Building on work from scholars like Tim Ingold, and strands of longstanding research in workplace studies and CSCW, our interest is in turning the attention to the active wo…
▽ More
In this short paper, we introduce work that is aiming to purposefully venture into this mesh of questions from a different starting point. Interjecting into the conversation, we want to ask: 'What is it to be skilled at work?' Building on work from scholars like Tim Ingold, and strands of longstanding research in workplace studies and CSCW, our interest is in turning the attention to the active work of 'being good', or 'being skilled', at what we as workers do. As we see it, skill provides a counterpoint to the version of intelligence that appears to be easily blackboxed in systems like Slack, and that ultimately reduces much of what people do to work well together. To put it slightly differently, skill - as we will argue below - gives us a way into thinking about work as a much more entangled endeavour, unfolding through multiple and interweaving sets of practices, places, tools and collaborations. In this vein, designing for the future of work seems to be about much more than where work is done or how we might bolt on discrete containers of intelligence. More fruitful would be attending to how we succeed in threading so many entities together to do our jobs well - in 'coming to be skilled'.
△ Less
Submitted 14 November, 2024;
originally announced November 2024.
-
"It's like a rubber duck that talks back": Understanding Generative AI-Assisted Data Analysis Workflows through a Participatory Prompting Study
Authors:
Ian Drosos,
Advait Sarkar,
Xiaotong Xu,
Carina Negreanu,
Sean Rintel,
Lev Tankelevitch
Abstract:
Generative AI tools can help users with many tasks. One such task is data analysis, which is notoriously challenging for non-expert end-users due to its expertise requirements, and where AI holds much potential, such as finding relevant data sources, proposing analysis strategies, and writing analysis code. To understand how data analysis workflows can be assisted or impaired by generative AI, we…
▽ More
Generative AI tools can help users with many tasks. One such task is data analysis, which is notoriously challenging for non-expert end-users due to its expertise requirements, and where AI holds much potential, such as finding relevant data sources, proposing analysis strategies, and writing analysis code. To understand how data analysis workflows can be assisted or impaired by generative AI, we conducted a study (n=15) using Bing Chat via participatory prompting. Participatory prompting is a recently developed methodology in which users and researchers reflect together on tasks through co-engagement with generative AI. In this paper we demonstrate the value of the participatory prompting method. We found that generative AI benefits the information foraging and sensemaking loops of data analysis in specific ways, but also introduces its own barriers and challenges, arising from the difficulties of query formulation, specifying context, and verifying results.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
The CoExplorer Technology Probe: A Generative AI-Powered Adaptive Interface to Support Intentionality in Planning and Running Video Meetings
Authors:
Gun Woo Warren Park,
Payod Panda,
Lev Tankelevitch,
Sean Rintel
Abstract:
Effective meetings are effortful, but traditional videoconferencing systems offer little support for reducing this effort across the meeting lifecycle. Generative AI (GenAI) has the potential to radically redefine meetings by augmenting intentional meeting behaviors. CoExplorer, our novel adaptive meeting prototype, preemptively generates likely phases that meetings would undergo, tools that allow…
▽ More
Effective meetings are effortful, but traditional videoconferencing systems offer little support for reducing this effort across the meeting lifecycle. Generative AI (GenAI) has the potential to radically redefine meetings by augmenting intentional meeting behaviors. CoExplorer, our novel adaptive meeting prototype, preemptively generates likely phases that meetings would undergo, tools that allow capturing attendees' thoughts before the meeting, and for each phase, window layouts, and appropriate applications and files. Using CoExplorer as a technology probe in a guided walkthrough, we studied its potential in a sample of participants from a global technology company. Our findings suggest that GenAI has the potential to help meetings stay on track and reduce workload, although concerns were raised about users' agency, trust, and possible disruption to traditional meeting norms. We discuss these concerns and their design implications for the development of GenAI meeting technology.
△ Less
Submitted 29 May, 2024; v1 submitted 28 May, 2024;
originally announced May 2024.
-
Mental Models of Meeting Goals: Supporting Intentionality in Meeting Technologies
Authors:
Ava Elizabeth Scott,
Lev Tankelevitch,
Sean Rintel
Abstract:
Ineffective meetings due to unclear goals are major obstacles to productivity, yet support for intentionality is surprisingly scant in our meeting and allied workflow technologies. To design for intentionality, we need to understand workers' attitudes and practices around goals. We interviewed 21 employees of a global technology company and identified contrasting mental models of meeting goals: me…
▽ More
Ineffective meetings due to unclear goals are major obstacles to productivity, yet support for intentionality is surprisingly scant in our meeting and allied workflow technologies. To design for intentionality, we need to understand workers' attitudes and practices around goals. We interviewed 21 employees of a global technology company and identified contrasting mental models of meeting goals: meetings as a means to an end, and meetings as an end in themselves. We explore how these mental models impact how meeting goals arise, goal prioritization, obstacles to considering goals, and how lack of alignment around goals may create tension between organizers and attendees. We highlight the challenges in balancing preparation, constraining scope, and clear outcomes, with the need for intentional adaptability and discovery in meetings. Our findings have implications for designing systems which increase effectiveness in meetings by catalyzing intentionality and reducing tension in the organisation of meetings.
△ Less
Submitted 28 February, 2024;
originally announced February 2024.
-
Ironies of Generative AI: Understanding and mitigating productivity loss in human-AI interactions
Authors:
Auste Simkute,
Lev Tankelevitch,
Viktor Kewenig,
Ava Elizabeth Scott,
Abigail Sellen,
Sean Rintel
Abstract:
Generative AI (GenAI) systems offer opportunities to increase user productivity in many tasks, such as programming and writing. However, while they boost productivity in some studies, many others show that users are working ineffectively with GenAI systems and losing productivity. Despite the apparent novelty of these usability challenges, these 'ironies of automation' have been observed for over…
▽ More
Generative AI (GenAI) systems offer opportunities to increase user productivity in many tasks, such as programming and writing. However, while they boost productivity in some studies, many others show that users are working ineffectively with GenAI systems and losing productivity. Despite the apparent novelty of these usability challenges, these 'ironies of automation' have been observed for over three decades in Human Factors research on the introduction of automation in domains such as aviation, automated driving, and intelligence. We draw on this extensive research alongside recent GenAI user studies to outline four key reasons for productivity loss with GenAI systems: a shift in users' roles from production to evaluation, unhelpful restructuring of workflows, interruptions, and a tendency for automation to make easy tasks easier and hard tasks harder. We then suggest how Human Factors research can also inform GenAI system design to mitigate productivity loss by using approaches such as continuous feedback, system personalization, ecological interface design, task stabilization, and clear task allocation. Thus, we ground developments in GenAI system usability in decades of Human Factors research, ensuring that the design of human-AI interactions in this rapidly moving field learns from history instead of repeating it.
△ Less
Submitted 17 February, 2024;
originally announced February 2024.
-
Participatory prompting: a user-centric research method for eliciting AI assistance opportunities in knowledge workflows
Authors:
Advait Sarkar,
Ian Drosos,
Rob Deline,
Andrew D. Gordon,
Carina Negreanu,
Sean Rintel,
Jack Williams,
Benjamin Zorn
Abstract:
Generative AI, such as image generation models and large language models, stands to provide tremendous value to end-user programmers in creative and knowledge workflows. Current research methods struggle to engage end-users in a realistic conversation that balances the actually existing capabilities of generative AI with the open-ended nature of user workflows and the many opportunities for the ap…
▽ More
Generative AI, such as image generation models and large language models, stands to provide tremendous value to end-user programmers in creative and knowledge workflows. Current research methods struggle to engage end-users in a realistic conversation that balances the actually existing capabilities of generative AI with the open-ended nature of user workflows and the many opportunities for the application of this technology. In this work-in-progress paper, we introduce participatory prompting, a method for eliciting opportunities for generative AI in end-user workflows. The participatory prompting method combines a contextual inquiry and a researcher-mediated interaction with a generative model, which helps study participants interact with a generative model without having to develop prompting strategies of their own. We discuss the ongoing development of a study whose aim will be to identify end-user programming opportunities for generative AI in data analysis workflows.
△ Less
Submitted 27 December, 2023;
originally announced December 2023.
-
The Metacognitive Demands and Opportunities of Generative AI
Authors:
Lev Tankelevitch,
Viktor Kewenig,
Auste Simkute,
Ava Elizabeth Scott,
Advait Sarkar,
Abigail Sellen,
Sean Rintel
Abstract:
Generative AI (GenAI) systems offer unprecedented opportunities for transforming professional and personal work, yet present challenges around prompting, evaluating and relying on outputs, and optimizing workflows. We argue that metacognition$\unicode{x2013}$the psychological ability to monitor and control one's thoughts and behavior$\unicode{x2013}$offers a valuable lens to understand and design…
▽ More
Generative AI (GenAI) systems offer unprecedented opportunities for transforming professional and personal work, yet present challenges around prompting, evaluating and relying on outputs, and optimizing workflows. We argue that metacognition$\unicode{x2013}$the psychological ability to monitor and control one's thoughts and behavior$\unicode{x2013}$offers a valuable lens to understand and design for these usability challenges. Drawing on research in psychology and cognitive science, and recent GenAI user studies, we illustrate how GenAI systems impose metacognitive demands on users, requiring a high degree of metacognitive monitoring and control. We propose these demands could be addressed by integrating metacognitive support strategies into GenAI systems, and by designing GenAI systems to reduce their metacognitive demand by targeting explainability and customizability. Metacognition offers a coherent framework for understanding the usability challenges posed by GenAI, and provides novel research and design directions to advance human-AI interaction.
△ Less
Submitted 12 March, 2024; v1 submitted 17 December, 2023;
originally announced December 2023.
-
What was Hybrid? A Systematic Review of Hybrid Collaboration and Meetings Research
Authors:
Thomas Neumayr,
Banu Saatci,
Sean Rintel,
Clemens Nylandsted Klokmose,
Mirjam Augstein
Abstract:
Interest in hybrid collaboration and meetings (HCM), where several co-located participants engage in coordinated work with remote participants, is gaining unprecedented momentum after the rapid shift in working from home due to the COVID-19 pandemic. However, while the interest is new, researchers have been exploring HCM phenomena for decades, albeit dispersed across diverse research traditions, u…
▽ More
Interest in hybrid collaboration and meetings (HCM), where several co-located participants engage in coordinated work with remote participants, is gaining unprecedented momentum after the rapid shift in working from home due to the COVID-19 pandemic. However, while the interest is new, researchers have been exploring HCM phenomena for decades, albeit dispersed across diverse research traditions, using different terms, definitions, and frameworks. In this article, we present a systematic literature review of the contexts and tools of HCM in the ACM Digital Library. We obtained approximately 1,200 results, which were narrowed down to 62 key articles. We report on the terms, citations, venues, authors, domains, study types, and data of these publications and present a taxonomic overview based on their reported hybrid settings' actual characteristics. We discuss why the SLR resulted in a relatively small number of publications, and then as a corollary, discuss how some excluded high-profile publications flesh out the SLR findings to provide important additional concepts. The SLR itself covers the ACM until November 2019, so our discussion also includes relevant 2020 and 2021 publications. The end result is a baseline that researchers and designers can use in shaping the post-COVID-19 future of HCM systems.
△ Less
Submitted 24 June, 2022; v1 submitted 11 November, 2021;
originally announced November 2021.
-
Bridging Social Distance During Social Distancing: Exploring Social Talk and Remote Collegiality in Video Conferencing
Authors:
Anna Bleakley,
Daniel Rough,
Justin Edwards,
Philip R. Doyle,
Odile Dumbleton,
Leigh Clark,
Sean Rintel,
Vincent Wade,
Benjamin R. Cowan
Abstract:
Video conferencing systems have long facilitated work-related conversations among remote teams. However, social distancing due to the COVID-19 pandemic has forced colleagues to use video conferencing platforms to additionally fulfil social needs. Social talk, or informal talk, is an important workplace practice that is used to build and maintain bonds in everyday interactions among colleagues. Cur…
▽ More
Video conferencing systems have long facilitated work-related conversations among remote teams. However, social distancing due to the COVID-19 pandemic has forced colleagues to use video conferencing platforms to additionally fulfil social needs. Social talk, or informal talk, is an important workplace practice that is used to build and maintain bonds in everyday interactions among colleagues. Currently, there is a limited understanding of how video conferencing facilitates multiparty social interactions among colleagues. In our paper, we examine social talk practices during the COVID-19 pandemic among remote colleagues through semi-structured interviews. We uncovered three key themes in our interviews, discussing 1) the changing purposes and opportunities afforded by using video conferencing for social talk with colleagues, 2) how the nature of existing relationships and status of colleagues influences social conversations and 3) the challenges and changing conversational norms around politeness and etiquette when using video conferencing to hold social conversations. We discuss these results in relation to the impact that video conferencing tools have on remote social talk between colleagues and outline design and best practice considerations for multiparty videoconferencing social talk in the workplace.
△ Less
Submitted 30 September, 2021;
originally announced September 2021.
-
Large Scale Analysis of Multitasking Behavior During Remote Meetings
Authors:
Hancheng Cao,
Chia-Jung Lee,
Shamsi Iqbal,
Mary Czerwinski,
Priscilla Wong,
Sean Rintel,
Brent Hecht,
Jaime Teevan,
Longqi Yang
Abstract:
Virtual meetings are critical for remote work because of the need for synchronous collaboration in the absence of in-person interactions. In-meeting multitasking is closely linked to people's productivity and wellbeing. However, we currently have limited understanding of multitasking in remote meetings and its potential impact. In this paper, we present what we believe is the most comprehensive st…
▽ More
Virtual meetings are critical for remote work because of the need for synchronous collaboration in the absence of in-person interactions. In-meeting multitasking is closely linked to people's productivity and wellbeing. However, we currently have limited understanding of multitasking in remote meetings and its potential impact. In this paper, we present what we believe is the most comprehensive study of remote meeting multitasking behavior through an analysis of a large-scale telemetry dataset collected from February to May 2020 of U.S. Microsoft employees and a 715-person diary study. Our results demonstrate that intrinsic meeting characteristics such as size, length, time, and type, significantly correlate with the extent to which people multitask, and multitasking can lead to both positive and negative outcomes. Our findings suggest important best-practice guidelines for remote meetings (e.g., avoid important meetings in the morning) and design implications for productivity tools (e.g., support positive remote multitasking).
△ Less
Submitted 28 January, 2021;
originally announced January 2021.
