-
Uncovering Factor Level Preferences to Improve Human-Model Alignment
Authors:
Juhyun Oh,
Eunsu Kim,
Jiseon Kim,
Wenda Xu,
Inha Cha,
William Yang Wang,
Alice Oh
Abstract:
Despite advancements in Large Language Model (LLM) alignment, understanding the reasons behind LLM preferences remains crucial for bridging the gap between desired and actual behavior. LLMs often exhibit biases or tendencies that diverge from human preferences, such as favoring certain writing styles or producing overly verbose outputs. However, current methods for evaluating preference alignment…
▽ More
Despite advancements in Large Language Model (LLM) alignment, understanding the reasons behind LLM preferences remains crucial for bridging the gap between desired and actual behavior. LLMs often exhibit biases or tendencies that diverge from human preferences, such as favoring certain writing styles or producing overly verbose outputs. However, current methods for evaluating preference alignment often lack explainability, relying on coarse-grained comparisons. To address this, we introduce PROFILE (PRObing Factors of InfLuence for Explainability), a novel framework that uncovers and quantifies the influence of specific factors driving preferences. PROFILE's factor level analysis explains the 'why' behind human-model alignment and misalignment, offering insights into the direction of model improvement. We apply PROFILE to analyze human and LLM preferences across three tasks: summarization, helpful response generation, and document-based question-answering. Our factor level analysis reveals a substantial discrepancy between human and LLM preferences in generation tasks, whereas LLMs show strong alignment with human preferences in evaluation tasks. We demonstrate how leveraging factor level insights, including addressing misaligned factors or exploiting the generation-evaluation gap, can improve alignment with human preferences. This work underscores the importance of explainable preference analysis and highlights PROFILE's potential to provide valuable training signals, driving further improvements in human-model alignment.
△ Less
Submitted 24 November, 2024; v1 submitted 9 October, 2024;
originally announced October 2024.
-
Ethics Pathways: A Design Activity for Reflecting on Ethics Engagement in HCI Research
Authors:
Inha Cha,
Ajit G. Pillai,
Richmond Y. Wong
Abstract:
This paper introduces Ethics Pathways, a design activity aimed at understanding HCI and design researchers' ethics engagements and flows during their research process. Despite a strong ethical commitment in these fields, challenges persist in grasping the complexity of researchers' engagement with ethics -- practices conducted to operationalize ethics -- in situated institutional contexts. Ethics…
▽ More
This paper introduces Ethics Pathways, a design activity aimed at understanding HCI and design researchers' ethics engagements and flows during their research process. Despite a strong ethical commitment in these fields, challenges persist in grasping the complexity of researchers' engagement with ethics -- practices conducted to operationalize ethics -- in situated institutional contexts. Ethics Pathways, developed through six playtesting sessions, offers a design approach to understanding the complexities of researchers' past ethics engagements in their work. This activity involves four main tasks: recalling ethical incidents; describing stakeholders involved in the situation; recounting their actions or speculative alternatives; and reflection and emotion walk-through. The paper reflects on the role of design decisions and facilitation strategies in achieving these goals. The design activity contributes to the discourse on ethical HCI research by conceptualizing ethics engagement as a part of ongoing research processing, highlighting connections between individual affective experiences, social interactions across power differences, and institutional goals.
△ Less
Submitted 26 May, 2024;
originally announced May 2024.
-
The Generative AI Paradox on Evaluation: What It Can Solve, It May Not Evaluate
Authors:
Juhyun Oh,
Eunsu Kim,
Inha Cha,
Alice Oh
Abstract:
This paper explores the assumption that Large Language Models (LLMs) skilled in generation tasks are equally adept as evaluators. We assess the performance of three LLMs and one open-source LM in Question-Answering (QA) and evaluation tasks using the TriviaQA (Joshi et al., 2017) dataset. Results indicate a significant disparity, with LLMs exhibiting lower performance in evaluation tasks compared…
▽ More
This paper explores the assumption that Large Language Models (LLMs) skilled in generation tasks are equally adept as evaluators. We assess the performance of three LLMs and one open-source LM in Question-Answering (QA) and evaluation tasks using the TriviaQA (Joshi et al., 2017) dataset. Results indicate a significant disparity, with LLMs exhibiting lower performance in evaluation tasks compared to generation tasks. Intriguingly, we discover instances of unfaithful evaluation where models accurately evaluate answers in areas where they lack competence, underscoring the need to examine the faithfulness and trustworthiness of LLMs as evaluators. This study contributes to the understanding of "the Generative AI Paradox" (West et al., 2023), highlighting a need to explore the correlation between generative excellence and evaluation proficiency, and the necessity to scrutinize the faithfulness aspect in model evaluations.
△ Less
Submitted 9 February, 2024;
originally announced February 2024.
-
The Grind for Good Data: Understanding ML Practitioners' Struggles and Aspirations in Making Good Data
Authors:
Inha Cha,
Juhyun Oh,
Cheul Young Park,
Jiyoon Han,
Hwalsuk Lee
Abstract:
We thought data to be simply given, but reality tells otherwise; it is costly, situation-dependent, and muddled with dilemmas, constantly requiring human intervention. The ML community's focus on quality data is increasing in the same vein, as good data is vital for successful ML systems. Nonetheless, few works have investigated the dataset builders and the specifics of what they do and struggle t…
▽ More
We thought data to be simply given, but reality tells otherwise; it is costly, situation-dependent, and muddled with dilemmas, constantly requiring human intervention. The ML community's focus on quality data is increasing in the same vein, as good data is vital for successful ML systems. Nonetheless, few works have investigated the dataset builders and the specifics of what they do and struggle to make good data. In this study, through semi-structured interviews with 19 ML experts, we present what humans actually do and consider in each step of the data construction pipeline. We further organize their struggles under three themes: 1) trade-offs from real-world constraints; 2) harmonizing assorted data workers for consistency; 3) the necessity of human intuition and tacit knowledge for processing data. Finally, we discuss why such struggles are inevitable for good data and what practitioners aspire, toward providing systematic support for data works.
△ Less
Submitted 27 November, 2022;
originally announced November 2022.
-
Secure Operations on Tree-Formed Verification Data
Authors:
Andreas U. Schmidt,
Andreas Leicher,
Yogendra Shah,
Inhyok Cha
Abstract:
We define secure operations with tree-formed, protected verification data registers. Functionality is conceptually added to Trusted Platform Modules (TPMs) to handle Platform Configuration Registers (PCRs) which represent roots of hash trees protecting the integrity of tree-formed Stored Measurement Logs (SMLs). This enables verification and update of an inner node of an SML and even attestation t…
▽ More
We define secure operations with tree-formed, protected verification data registers. Functionality is conceptually added to Trusted Platform Modules (TPMs) to handle Platform Configuration Registers (PCRs) which represent roots of hash trees protecting the integrity of tree-formed Stored Measurement Logs (SMLs). This enables verification and update of an inner node of an SML and even attestation to its value with the same security level as for ordinary PCRs. As an important application, it is shown how certification of SML subtrees enables attestation of platform properties.
△ Less
Submitted 19 August, 2010;
originally announced August 2010.
-
Tree-formed Verification Data for Trusted Platforms
Authors:
Andreas U. Schmidt,
Andreas Leicher,
Yogendra Shah,
Inhyok Cha
Abstract:
The establishment of trust relationships to a computing platform relies on validation processes. Validation allows an external entity to build trust in the expected behaviour of the platform based on provided evidence of the platform's configuration. In a process like remote attestation, the 'trusted' platform submits verification data created during a start up process. These data consist of hardw…
▽ More
The establishment of trust relationships to a computing platform relies on validation processes. Validation allows an external entity to build trust in the expected behaviour of the platform based on provided evidence of the platform's configuration. In a process like remote attestation, the 'trusted' platform submits verification data created during a start up process. These data consist of hardware-protected values of platform configuration registers, containing nested measurement values, e.g., hash values, of loaded or started components. Commonly, the register values are created in linear order by a hardware-secured operation. Fine-grained diagnosis of components, based on the linear order of verification data and associated measurement logs, is not optimal. We propose a method to use tree-formed verification data to validate a platform. Component measurement values represent leaves, and protected registers represent roots of a hash tree. We describe the basic mechanism of validating a platform using tree-formed measurement logs and root registers and show an logarithmic speed-up for the search of faults. Secure creation of a tree is possible using a limited number of hardware-protected registers and a single protected operation. In this way, the security of tree-formed verification data is maintained.
△ Less
Submitted 18 October, 2012; v1 submitted 5 July, 2010;
originally announced July 2010.
