-
Justified Evidence Collection for Argument-based AI Fairness Assurance
Authors:
Alpay Sabuncuoglu,
Christopher Burr,
Carsten Maple
Abstract:
It is well recognised that ensuring fair AI systems is a complex sociotechnical challenge, which requires careful deliberation and continuous oversight across all stages of a system's lifecycle, from defining requirements to model deployment and deprovisioning. Dynamic argument-based assurance cases, which present structured arguments supported by evidence, have emerged as a systematic approach to…
▽ More
It is well recognised that ensuring fair AI systems is a complex sociotechnical challenge, which requires careful deliberation and continuous oversight across all stages of a system's lifecycle, from defining requirements to model deployment and deprovisioning. Dynamic argument-based assurance cases, which present structured arguments supported by evidence, have emerged as a systematic approach to evaluating and mitigating safety risks and hazards in AI-enabled system development and have also been extended to deal with broader normative goals such as fairness and explainability. This paper introduces a systems-engineering-driven framework, supported by software tooling, to operationalise a dynamic approach to argument-based assurance in two stages. In the first stage, during the requirements planning phase, a multi-disciplinary and multi-stakeholder team define goals and claims to be established (and evidenced) by conducting a comprehensive fairness governance process. In the second stage, a continuous monitoring interface gathers evidence from existing artefacts (e.g. metrics from automated tests), such as model, data, and use case documentation, to support these arguments dynamically. The framework's effectiveness is demonstrated through an illustrative case study in finance, with a focus on supporting fairness-related arguments.
△ Less
Submitted 12 May, 2025;
originally announced May 2025.
-
Towards Building Child-Centered Machine Learning Pipelines: Use Cases from K-12 and Higher-Education
Authors:
Alpay Sabuncuoglu,
Ceylan Besevli,
T. Metin Sezgin
Abstract:
Researchers and policy-makers have started creating frameworks and guidelines for building machine-learning (ML) pipelines with a human-centered lens. Machine Learning pipelines stand for all the necessary steps to develop ML systems (e.g., developing a predictive keyboard). On the other hand, a child-centered focus in developing ML systems has been recently gaining interest as children are becomi…
▽ More
Researchers and policy-makers have started creating frameworks and guidelines for building machine-learning (ML) pipelines with a human-centered lens. Machine Learning pipelines stand for all the necessary steps to develop ML systems (e.g., developing a predictive keyboard). On the other hand, a child-centered focus in developing ML systems has been recently gaining interest as children are becoming users of these products. These efforts dominantly focus on children's interaction with ML-based systems. However, from our experience, ML pipelines are yet to be adapted using a child-centered lens. In this paper, we list the questions we ask ourselves in adapting human-centered ML pipelines to child-centered ones. We also summarize two case studies of building end-to-end ML pipelines for children's products.
△ Less
Submitted 19 April, 2023;
originally announced April 2023.
-
Multimodal Group Activity Dataset for Classroom Engagement Level Prediction
Authors:
Alpay Sabuncuoglu,
T. Metin Sezgin
Abstract:
We collected a new dataset that includes approximately eight hours of audiovisual recordings of a group of students and their self-evaluation scores for classroom engagement. The dataset and data analysis scripts are available on our open-source repository. We developed baseline face-based and group-activity-based image and video recognition models. Our image models yield 45-85% test accuracy with…
▽ More
We collected a new dataset that includes approximately eight hours of audiovisual recordings of a group of students and their self-evaluation scores for classroom engagement. The dataset and data analysis scripts are available on our open-source repository. We developed baseline face-based and group-activity-based image and video recognition models. Our image models yield 45-85% test accuracy with face-area inputs on person-based classification task. Our video models achieved up to 71% test accuracy on group-level prediction using group activity video inputs. In this technical report, we shared the details of our end-to-end human-centered engagement analysis pipeline from data collection to model development.
△ Less
Submitted 18 April, 2023;
originally announced April 2023.
-
Exploring Children's Use of Self-Made Tangibles in Programming
Authors:
Alpay Sabuncuoglu,
T. Metin Sezgin
Abstract:
Defining abstract algorithmic structures like functions and variables using self-made tangibles can enhance the usability and affordability of the tangible programming experience by maintaining the input modality and physical interaction throughout the activity and reducing the dependence on electronic devices. However, existing tangible programming environments use digital interfaces to save abst…
▽ More
Defining abstract algorithmic structures like functions and variables using self-made tangibles can enhance the usability and affordability of the tangible programming experience by maintaining the input modality and physical interaction throughout the activity and reducing the dependence on electronic devices. However, existing tangible programming environments use digital interfaces to save abstract definitions such as functions and variables, as designing new tangibles is challenging for children due to their limited experience using abstract definitions. We conducted a series of design workshops with children to understand their self-made tangible creation abilities and develop design considerations for tangible computing such as paper programming.
This paper reports:
1) Our insights on how students conceptualize and design tangible programming blocks,
2) Design considerations of self-made tangibles to yield higher understandability and memorability,
3) Tests of our design considerations in creating self-made tangibles in real-life coding activities.
△ Less
Submitted 12 October, 2022;
originally announced October 2022.
-
Teaching K-12 Classrooms Data Programming: A Three-Week Workshop with Online and Unplugged Activities
Authors:
Alpay Sabuncuoğlu,
Asım Evren Yantaç,
Tevfik Metin Sezgin
Abstract:
This paper shares our experience in a three-session online workshop using a new web-based data programming environment, Marti. The programming environment uses a card-based programming strategy in both unplugged and online activities. Educators can use the physical cards in a board-game style or use the programming environment's mobile application to scan these cards and render the final visualiza…
▽ More
This paper shares our experience in a three-session online workshop using a new web-based data programming environment, Marti. The programming environment uses a card-based programming strategy in both unplugged and online activities. Educators can use the physical cards in a board-game style or use the programming environment's mobile application to scan these cards and render the final visualization on their phones/tablets. The web environment also uses visual draggable cards for programming that can manipulate and visualize data. We used Marti and its offered unplugged activities in three sessions with 12 middle school and 12 high school students, focusing on the data fundamentals, analysis, and visualization. We assert that integrating unplugged-style pseudo-code creation and supporting a similar experience using the available devices have considerable potential for delivering equal and affordable data programming education for all.
△ Less
Submitted 11 October, 2021;
originally announced October 2021.
