Showing 1–47 of 47 results for author: Kwon, C

USPR: Learning a Unified Solver for Profiled Routing

Authors: Chuanbo Hua, Federico Berto, Zhikai Zhao, Jiwoo Son, Changhyun Kwon, Jinkyoo Park

Abstract: The Profiled Vehicle Routing Problem (PVRP) extends the classical VRP by incorporating vehicle-client-specific preferences and constraints, reflecting real-world requirements such as zone restrictions and service-level preferences. While recent reinforcement learning (RL) solvers have shown promise, they require retraining for each new profile distribution, suffer from poor representation ability,… ▽ More The Profiled Vehicle Routing Problem (PVRP) extends the classical VRP by incorporating vehicle-client-specific preferences and constraints, reflecting real-world requirements such as zone restrictions and service-level preferences. While recent reinforcement learning (RL) solvers have shown promise, they require retraining for each new profile distribution, suffer from poor representation ability, and struggle to generalize to out-of-distribution instances. In this paper, we address these limitations by introducing USPR (Unified Solver for Profiled Routing), a novel framework that natively handles arbitrary profile types. USPR introduces three key innovations: (i) Profile Embeddings (PE) to encode any combination of profile types; (ii) Multi-Head Profiled Attention (MHPA), an attention mechanism that models rich interactions between vehicles and clients; (iii) Profile-aware Score Reshaping (PSR), which dynamically adjusts decoder logits using profile scores to improve generalization. Empirical results on diverse PVRP benchmarks demonstrate that USPR achieves state-of-the-art results among learning-based methods while offering significant gains in flexibility and computational efficiency. We make our source code publicly available to foster future research at https://github.com/ai4co/uspr. △ Less

Submitted 8 May, 2025; originally announced May 2025.

The Iterative Chainlet Partitioning Algorithm for the Traveling Salesman Problem with Drone and Neural Acceleration

Authors: Jae Hyeok Lee, Minjun Kim, Jinkyoo Park, Changhyun Kwon

Abstract: This study introduces the Iterative Chainlet Partitioning (ICP) algorithm and its neural acceleration for solving the Traveling Salesman Problem with Drone (TSP-D). The proposed ICP algorithm decomposes a TSP-D solution into smaller segments called chainlets, each optimized individually by a dynamic programming subroutine. The chainlet with the highest improvement is updated and the procedure is r… ▽ More This study introduces the Iterative Chainlet Partitioning (ICP) algorithm and its neural acceleration for solving the Traveling Salesman Problem with Drone (TSP-D). The proposed ICP algorithm decomposes a TSP-D solution into smaller segments called chainlets, each optimized individually by a dynamic programming subroutine. The chainlet with the highest improvement is updated and the procedure is repeated until no further improvement is possible. The number of subroutine calls is bounded linearly in problem size for the first iteration and remains constant in subsequent iterations, ensuring algorithmic scalability. Empirical results show that ICP outperforms existing algorithms in both solution quality and computational time. Tested over 1,059 benchmark instances, ICP yields an average improvement of 2.75% in solution quality over the previous state-of-the-art algorithm while reducing computational time by 79.8%. The procedure is deterministic, ensuring reliability without requiring multiple runs. The subroutine is the computational bottleneck in the already efficient ICP algorithm. To reduce the necessity of subroutine calls, we integrate a graph neural network (GNN) to predict incremental improvements. We demonstrate that the resulting Neuro ICP (NICP) achieves substantial acceleration while maintaining solution quality. Compared to ICP, NICP reduces the total computational time by 49.7%, while the objective function value increase is limited to 0.12%. The framework's adaptability to various operational constraints makes it a valuable foundation for developing efficient algorithms for truck-drone synchronized routing problems. △ Less

Submitted 21 April, 2025; originally announced April 2025.

CP-AgentNet: Autonomous and Explainable Communication Protocol Design Using Generative Agents

Authors: Dae Cheol Kwon, Xinyu Zhang

Abstract: Although DRL (deep reinforcement learning) has emerged as a powerful tool for making better decisions than existing hand-crafted communication protocols, it faces significant limitations: 1) Selecting the appropriate neural network architecture and setting hyperparameters are crucial for achieving desired performance levels, requiring domain expertise. 2) The decision-making process in DRL models… ▽ More Although DRL (deep reinforcement learning) has emerged as a powerful tool for making better decisions than existing hand-crafted communication protocols, it faces significant limitations: 1) Selecting the appropriate neural network architecture and setting hyperparameters are crucial for achieving desired performance levels, requiring domain expertise. 2) The decision-making process in DRL models is often opaque, commonly described as a 'black box.' 3) DRL models are data hungry. In response, we propose CP-AgentNet, the first framework designed to use generative agents for developing communication network protocols. This approach addresses these challenges by creating an autonomous system for protocol design, significantly reducing human effort. We developed LLMA (LLM-agents-based multiple access) and CPTCP (CP-Agent-based TCP) for heterogeneous environments. Our comprehensive simulations have demonstrated the efficient coexistence of LLMA and CPTCP with nodes using different types of protocols, as well as enhanced explainability. △ Less

Submitted 22 March, 2025; originally announced March 2025.

Neural Combinatorial Optimization for Real-World Routing

Authors: Jiwoo Son, Zhikai Zhao, Federico Berto, Chuanbo Hua, Changhyun Kwon, Jinkyoo Park

Abstract: Vehicle Routing Problems (VRPs) are a class of NP-hard problems ubiquitous in several real-world logistics scenarios that pose significant challenges for optimization. Neural Combinatorial Optimization (NCO) has emerged as a promising alternative to classical approaches, as it can learn fast heuristics to solve VRPs. However, most research works in NCO for VRPs focus on simplified settings, which… ▽ More Vehicle Routing Problems (VRPs) are a class of NP-hard problems ubiquitous in several real-world logistics scenarios that pose significant challenges for optimization. Neural Combinatorial Optimization (NCO) has emerged as a promising alternative to classical approaches, as it can learn fast heuristics to solve VRPs. However, most research works in NCO for VRPs focus on simplified settings, which do not account for asymmetric distances and travel durations that cannot be derived by simple Euclidean distances and unrealistic data distributions, hindering real-world deployment. This work introduces RRNCO (Real Routing NCO) to bridge the gap of NCO between synthetic and real-world VRPs in the critical aspects of both data and modeling. First, we introduce a new, openly available dataset with real-world data containing a diverse dataset of locations, distances, and duration matrices from 100 cities, considering realistic settings with actual routing distances and durations obtained from Open Source Routing Machine (OSRM). Second, we propose a novel approach that efficiently processes both node and edge features through contextual gating, enabling the construction of more informed node embedding, and we finally incorporate an Adaptation Attention Free Module (AAFM) with neural adaptive bias mechanisms that effectively integrates not only distance matrices but also angular relationships between nodes, allowing our model to capture rich structural information. RRNCO achieves state-of-the-art results in real-world VRPs among NCO methods. We make our dataset and code publicly available at https://github.com/ai4co/real-routing-nco. △ Less

Submitted 20 March, 2025; originally announced March 2025.

A Critical Analysis of the Usage of Dimensionality Reduction in Four Domains

Authors: Dylan Cashman, Mark Keller, Hyeon Jeon, Bum Chul Kwon, Qianwen Wang

Abstract: Dimensionality reduction is used as an important tool for unraveling the complexities of high-dimensional datasets in many fields of science, such as cell biology, chemical informatics, and physics. Visualizations of the dimensionally reduced data enable scientists to delve into the intrinsic structures of their datasets and align them with established hypotheses. Visualization researchers have th… ▽ More Dimensionality reduction is used as an important tool for unraveling the complexities of high-dimensional datasets in many fields of science, such as cell biology, chemical informatics, and physics. Visualizations of the dimensionally reduced data enable scientists to delve into the intrinsic structures of their datasets and align them with established hypotheses. Visualization researchers have thus proposed many dimensionality reduction methods and interactive systems designed to uncover latent structures. At the same time, different scientific domains have formulated guidelines or common workflows for using dimensionality reduction techniques and visualizations for their respective fields. In this work, we present a critical analysis of the usage of dimensionality reduction in scientific domains outside of computer science. First, we conduct a bibliometric analysis of 21,249 academic publications that use dimensionality reduction to observe differences in the frequency of techniques across fields. Next, we conduct a survey of a 71-paper sample from four fields: biology, chemistry, physics, and business. Through this survey, we uncover common workflows, processes, and usage patterns, including the mixed use of confirmatory data analysis to validate a dataset and projection method and exploratory data analysis to then generate more hypotheses. We also find that misinterpretations and inappropriate usage is common, particularly in the visual interpretation of the resulting dimensionally reduced view. Lastly, we compare our observations with recent works in the visualization community in order to match work within our community to potential areas of impact outside our community. △ Less

Submitted 11 March, 2025; originally announced March 2025.

Comments: In submission to TVCG. Currently under minor revision

Neural Genetic Search in Discrete Spaces

Authors: Hyeonah Kim, Sanghyeok Choi, Jiwoo Son, Jinkyoo Park, Changhyun Kwon

Abstract: Effective search methods are crucial for improving the performance of deep generative models at test time. In this paper, we introduce a novel test-time search method, Neural Genetic Search (NGS), which incorporates the evolutionary mechanism of genetic algorithms into the generation procedure of deep models. The core idea behind NGS is its crossover, which is defined as parent-conditioned generat… ▽ More Effective search methods are crucial for improving the performance of deep generative models at test time. In this paper, we introduce a novel test-time search method, Neural Genetic Search (NGS), which incorporates the evolutionary mechanism of genetic algorithms into the generation procedure of deep models. The core idea behind NGS is its crossover, which is defined as parent-conditioned generation using trained generative models. This approach offers a versatile and easy-to-implement search algorithm for deep generative models. We demonstrate the effectiveness and flexibility of NGS through experiments across three distinct domains: routing problems, adversarial prompt generation for language models, and molecular design. △ Less

Submitted 16 June, 2025; v1 submitted 8 February, 2025; originally announced February 2025.

Comments: ICML 2025

CAMP: Collaborative Attention Model with Profiles for Vehicle Routing Problems

Authors: Chuanbo Hua, Federico Berto, Jiwoo Son, Seunghyun Kang, Changhyun Kwon, Jinkyoo Park

Abstract: The profiled vehicle routing problem (PVRP) is a generalization of the heterogeneous capacitated vehicle routing problem (HCVRP) in which the objective is to optimize the routes of vehicles to serve client demands subject to different vehicle profiles, with each having a preference or constraint on a per-client basis. While existing learning methods have shown promise for solving the HCVRP in real… ▽ More The profiled vehicle routing problem (PVRP) is a generalization of the heterogeneous capacitated vehicle routing problem (HCVRP) in which the objective is to optimize the routes of vehicles to serve client demands subject to different vehicle profiles, with each having a preference or constraint on a per-client basis. While existing learning methods have shown promise for solving the HCVRP in real-time, no learning method exists to solve the more practical and challenging PVRP. In this paper, we propose a Collaborative Attention Model with Profiles (CAMP), a novel approach that learns efficient solvers for PVRP using multi-agent reinforcement learning. CAMP employs a specialized attention-based encoder architecture to embed profiled client embeddings in parallel for each vehicle profile. We design a communication layer between agents for collaborative decision-making across profiled embeddings at each decoding step and a batched pointer mechanism to attend to the profiled embeddings to evaluate the likelihood of the next actions. We evaluate CAMP on two variants of PVRPs: PVRP with preferences, which explicitly influence the reward function, and PVRP with zone constraints with different numbers of agents and clients, demonstrating that our learned solvers achieve competitive results compared to both classical state-of-the-art neural multi-agent models in terms of solution quality and computational efficiency. We make our code openly available at https://github.com/ai4co/camp. △ Less

Submitted 4 February, 2025; v1 submitted 6 January, 2025; originally announced January 2025.

Comments: Accepted at AAMAS 2025

Multi-view biomedical foundation models for molecule-target and property prediction

Authors: Parthasarathy Suryanarayanan, Yunguang Qiu, Shreyans Sethi, Diwakar Mahajan, Hongyang Li, Yuxin Yang, Elif Eyigoz, Aldo Guzman Saenz, Daniel E. Platt, Timothy H. Rumbell, Kenney Ng, Sanjoy Dey, Myson Burch, Bum Chul Kwon, Pablo Meyer, Feixiong Cheng, Jianying Hu, Joseph A. Morrone

Abstract: Foundation models applied to bio-molecular space hold promise to accelerate drug discovery. Molecular representation is key to building such models. Previous works have typically focused on a single representation or view of the molecules. Here, we develop a multi-view foundation model approach, that integrates molecular views of graph, image and text. Single-view foundation models are each pre-tr… ▽ More Foundation models applied to bio-molecular space hold promise to accelerate drug discovery. Molecular representation is key to building such models. Previous works have typically focused on a single representation or view of the molecules. Here, we develop a multi-view foundation model approach, that integrates molecular views of graph, image and text. Single-view foundation models are each pre-trained on a dataset of up to 200M molecules and then aggregated into combined representations. Our multi-view model is validated on a diverse set of 18 tasks, encompassing ligand-protein binding, molecular solubility, metabolism and toxicity. We show that the multi-view models perform robustly and are able to balance the strengths and weaknesses of specific views. We then apply this model to screen compounds against a large (>100 targets) set of G Protein-Coupled receptors (GPCRs). From this library of targets, we identify 33 that are related to Alzheimer's disease. On this subset, we employ our model to identify strong binders, which are validated through structure-based modeling and identification of key binding motifs. △ Less

Submitted 31 January, 2025; v1 submitted 25 October, 2024; originally announced October 2024.

Comments: 37 pages including supplement. 10 figures, 8 tables

Parallel AutoRegressive Models for Multi-Agent Combinatorial Optimization

Authors: Federico Berto, Chuanbo Hua, Laurin Luttmann, Jiwoo Son, Junyoung Park, Kyuree Ahn, Changhyun Kwon, Lin Xie, Jinkyoo Park

Abstract: Combinatorial optimization problems involving multiple agents are notoriously challenging due to their NP-hard nature and the necessity for effective agent coordination. Despite advancements in learning-based methods, existing approaches often face critical limitations, including suboptimal agent coordination, poor generalizability, and high computational latency. To address these issues, we propo… ▽ More Combinatorial optimization problems involving multiple agents are notoriously challenging due to their NP-hard nature and the necessity for effective agent coordination. Despite advancements in learning-based methods, existing approaches often face critical limitations, including suboptimal agent coordination, poor generalizability, and high computational latency. To address these issues, we propose Parallel AutoRegressive Combinatorial Optimization (PARCO), a reinforcement learning framework designed to construct high-quality solutions for multi-agent combinatorial tasks efficiently. To this end, PARCO integrates three key components: (1) transformer-based communication layers to enable effective agent collaboration during parallel solution construction, (2) a multiple pointer mechanism for low-latency, parallel agent decision-making, and (3) priority-based conflict handlers to resolve decision conflicts via learned priorities. We evaluate PARCO in multi-agent vehicle routing and scheduling problems where our approach outperforms state-of-the-art learning methods and demonstrates strong generalization ability and remarkable computational efficiency. Code available at: https://github.com/ai4co/parco. △ Less

Submitted 5 February, 2025; v1 submitted 5 September, 2024; originally announced September 2024.

DG Comics: Semi-Automatically Authoring Graph Comics for Dynamic Graphs

Authors: Joohee Kim, Hyunwook Lee, Duc M. Nguyen, Minjeong Shin, Bum Chul Kwon, Sungahn Ko, Niklas Elmqvist

Abstract: Comics are an effective method for sequential data-driven storytelling, especially for dynamic graphs -- graphs whose vertices and edges change over time. However, manually creating such comics is currently time-consuming, complex, and error-prone. In this paper, we propose DG Comics, a novel comic authoring tool for dynamic graphs that allows users to semi-automatically build and annotate comics.… ▽ More Comics are an effective method for sequential data-driven storytelling, especially for dynamic graphs -- graphs whose vertices and edges change over time. However, manually creating such comics is currently time-consuming, complex, and error-prone. In this paper, we propose DG Comics, a novel comic authoring tool for dynamic graphs that allows users to semi-automatically build and annotate comics. The tool uses a newly developed hierarchical clustering algorithm to segment consecutive snapshots of dynamic graphs while preserving their chronological order. It also presents rich information on both individuals and communities extracted from dynamic graphs in multiple views, where users can explore dynamic graphs and choose what to tell in comics. For evaluation, we provide an example and report the results of a user study and an expert review. △ Less

Submitted 9 August, 2024; originally announced August 2024.

Comments: To appear in IEEE Transactions on Visualization and Computer Graphics

MiMICRI: Towards Domain-centered Counterfactual Explanations of Cardiovascular Image Classification Models

Authors: Grace Guo, Lifu Deng, Animesh Tandon, Alex Endert, Bum Chul Kwon

Abstract: The recent prevalence of publicly accessible, large medical imaging datasets has led to a proliferation of artificial intelligence (AI) models for cardiovascular image classification and analysis. At the same time, the potentially significant impacts of these models have motivated the development of a range of explainable AI (XAI) methods that aim to explain model predictions given certain image i… ▽ More The recent prevalence of publicly accessible, large medical imaging datasets has led to a proliferation of artificial intelligence (AI) models for cardiovascular image classification and analysis. At the same time, the potentially significant impacts of these models have motivated the development of a range of explainable AI (XAI) methods that aim to explain model predictions given certain image inputs. However, many of these methods are not developed or evaluated with domain experts, and explanations are not contextualized in terms of medical expertise or domain knowledge. In this paper, we propose a novel framework and python library, MiMICRI, that provides domain-centered counterfactual explanations of cardiovascular image classification models. MiMICRI helps users interactively select and replace segments of medical images that correspond to morphological structures. From the counterfactuals generated, users can then assess the influence of each segment on model predictions, and validate the model against known medical facts. We evaluate this library with two medical experts. Our evaluation demonstrates that a domain-centered XAI approach can enhance the interpretability of model explanations, and help experts reason about models in terms of relevant domain knowledge. However, concerns were also surfaced about the clinical plausibility of the counterfactuals generated. We conclude with a discussion on the generalizability and trustworthiness of the MiMICRI framework, as well as the implications of our findings on the development of domain-centered XAI methods for model interpretability in healthcare contexts. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 14 pages, 6 figures, ACM FAccT 2024

ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale

Authors: Jinbin Huang, Chen Chen, Aditi Mishra, Bum Chul Kwon, Zhicheng Liu, Chris Bryan

Abstract: Generative image models have emerged as a promising technology to produce realistic images. Despite potential benefits, concerns grow about its misuse, particularly in generating deceptive images that could raise significant ethical, legal, and societal issues. Consequently, there is growing demand to empower users to effectively discern and comprehend patterns of AI-generated images. To this end,… ▽ More Generative image models have emerged as a promising technology to produce realistic images. Despite potential benefits, concerns grow about its misuse, particularly in generating deceptive images that could raise significant ethical, legal, and societal issues. Consequently, there is growing demand to empower users to effectively discern and comprehend patterns of AI-generated images. To this end, we developed ASAP, an interactive visualization system that automatically extracts distinct patterns of AI-generated images and allows users to interactively explore them via various views. To uncover fake patterns, ASAP introduces a novel image encoder, adapted from CLIP, which transforms images into compact "distilled" representations, enriched with information for differentiating authentic and fake images. These representations generate gradients that propagate back to the attention maps of CLIP's transformer block. This process quantifies the relative importance of each pixel to image authenticity or fakeness, exposing key deceptive patterns. ASAP enables the at scale interactive analysis of these patterns through multiple, coordinated visualizations. This includes a representation overview with innovative cell glyphs to aid in the exploration and qualitative evaluation of fake patterns across a vast array of images, as well as a pattern view that displays authenticity-indicating patterns in images and quantifies their impact. ASAP supports the analysis of cutting-edge generative models with the latest architectures, including GAN-based models like proGAN and diffusion models like the latent diffusion model. We demonstrate ASAP's usefulness through two usage scenarios using multiple fake image detection benchmark datasets, revealing its ability to identify and understand hidden patterns in AI-generated images, especially in detecting fake human faces produced by diffusion-based techniques. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: 9 pages, 6 figures

Fully autonomous tuning of a spin qubit

Authors: Jonas Schuff, Miguel J. Carballido, Madeleine Kotzagiannidis, Juan Carlos Calvo, Marco Caselli, Jacob Rawling, David L. Craig, Barnaby van Straaten, Brandon Severin, Federico Fedele, Simon Svab, Pierre Chevalier Kwon, Rafael S. Eggli, Taras Patlatiuk, Nathan Korda, Dominik Zumbühl, Natalia Ares

Abstract: Spanning over two decades, the study of qubits in semiconductors for quantum computing has yielded significant breakthroughs. However, the development of large-scale semiconductor quantum circuits is still limited by challenges in efficiently tuning and operating these circuits. Identifying optimal operating conditions for these qubits is complex, involving the exploration of vast parameter spaces… ▽ More Spanning over two decades, the study of qubits in semiconductors for quantum computing has yielded significant breakthroughs. However, the development of large-scale semiconductor quantum circuits is still limited by challenges in efficiently tuning and operating these circuits. Identifying optimal operating conditions for these qubits is complex, involving the exploration of vast parameter spaces. This presents a real 'needle in the haystack' problem, which, until now, has resisted complete automation due to device variability and fabrication imperfections. In this study, we present the first fully autonomous tuning of a semiconductor qubit, from a grounded device to Rabi oscillations, a clear indication of successful qubit operation. We demonstrate this automation, achieved without human intervention, in a Ge/Si core/shell nanowire device. Our approach integrates deep learning, Bayesian optimization, and computer vision techniques. We expect this automation algorithm to apply to a wide range of semiconductor qubit devices, allowing for statistical studies of qubit quality metrics. As a demonstration of the potential of full automation, we characterise how the Rabi frequency and g-factor depend on barrier gate voltages for one of the qubits found by the algorithm. Twenty years after the initial demonstrations of spin qubit operation, this significant advancement is poised to finally catalyze the operation of large, previously unexplored quantum circuits. △ Less

Submitted 6 February, 2024; originally announced February 2024.

Latent Space Explorer: Visual Analytics for Multimodal Latent Space Exploration

Authors: Bum Chul Kwon, Samuel Friedman, Kai Xu, Steven A Lubitz, Anthony Philippakis, Puneet Batra, Patrick T Ellinor, Kenney Ng

Abstract: Machine learning models built on training data with multiple modalities can reveal new insights that are not accessible through unimodal datasets. For example, cardiac magnetic resonance images (MRIs) and electrocardiograms (ECGs) are both known to capture useful information about subjects' cardiovascular health status. A multimodal machine learning model trained from large datasets can potentiall… ▽ More Machine learning models built on training data with multiple modalities can reveal new insights that are not accessible through unimodal datasets. For example, cardiac magnetic resonance images (MRIs) and electrocardiograms (ECGs) are both known to capture useful information about subjects' cardiovascular health status. A multimodal machine learning model trained from large datasets can potentially predict the onset of heart-related diseases and provide novel medical insights about the cardiovascular system. Despite the potential benefits, it is difficult for medical experts to explore multimodal representation models without visual aids and to test the predictive performance of the models on various subpopulations. To address the challenges, we developed a visual analytics system called Latent Space Explorer. Latent Space Explorer provides interactive visualizations that enable users to explore the multimodal representation of subjects, define subgroups of interest, interactively decode data with different modalities with the selected subjects, and inspect the accuracy of the embedding in downstream prediction tasks. A user study was conducted with medical experts and their feedback provided useful insights into how Latent Space Explorer can help their analysis and possible new direction for further development in the medical domain. △ Less

Submitted 1 December, 2023; originally announced December 2023.

Comments: 7 pages, 5 figures

Genetic Algorithms with Neural Cost Predictor for Solving Hierarchical Vehicle Routing Problems

Authors: Abhay Sobhanan, Junyoung Park, Jinkyoo Park, Changhyun Kwon

Abstract: When vehicle routing decisions are intertwined with higher-level decisions, the resulting optimization problems pose significant challenges for computation. Examples are the multi-depot vehicle routing problem (MDVRP), where customers are assigned to depots before delivery, and the capacitated location routing problem (CLRP), where the locations of depots should be determined first. A simple and s… ▽ More When vehicle routing decisions are intertwined with higher-level decisions, the resulting optimization problems pose significant challenges for computation. Examples are the multi-depot vehicle routing problem (MDVRP), where customers are assigned to depots before delivery, and the capacitated location routing problem (CLRP), where the locations of depots should be determined first. A simple and straightforward approach for such hierarchical problems would be to separate the higher-level decisions from the complicated vehicle routing decisions. For each higher-level decision candidate, we may evaluate the underlying vehicle routing problems to assess the candidate. As this approach requires solving vehicle routing problems multiple times, it has been regarded as impractical in most cases. We propose a novel deep-learning-based approach called Genetic Algorithm with Neural Cost Predictor (GANCP) to tackle the challenge and simplify algorithm developments. For each higher-level decision candidate, we predict the objective function values of the underlying vehicle routing problems using a pre-trained graph neural network without actually solving the routing problems. In particular, our proposed neural network learns the objective values of the HGS-CVRP open-source package that solves capacitated vehicle routing problems. Our numerical experiments show that this simplified approach is effective and efficient in generating high-quality solutions for both MDVRP and CLRP and has the potential to expedite algorithm developments for complicated hierarchical problems. We provide computational results evaluated in the standard benchmark instances used in the literature. △ Less

Submitted 7 September, 2024; v1 submitted 21 October, 2023; originally announced October 2023.

People's Perceptions Toward Bias and Related Concepts in Large Language Models: A Systematic Review

Authors: Lu Wang, Max Song, Rezvaneh Rezapour, Bum Chul Kwon, Jina Huh-Yoo

Abstract: Large language models (LLMs) have brought breakthroughs in tasks including translation, summarization, information retrieval, and language generation, gaining growing interest in the CHI community. Meanwhile, the literature shows researchers' controversial perceptions about the efficacy, ethics, and intellectual abilities of LLMs. However, we do not know how people perceive LLMs that are pervasive… ▽ More Large language models (LLMs) have brought breakthroughs in tasks including translation, summarization, information retrieval, and language generation, gaining growing interest in the CHI community. Meanwhile, the literature shows researchers' controversial perceptions about the efficacy, ethics, and intellectual abilities of LLMs. However, we do not know how people perceive LLMs that are pervasive in everyday tools, specifically regarding their experience with LLMs around bias, stereotypes, social norms, or safety. In this study, we conducted a systematic review to understand what empirical insights papers have gathered about people's perceptions toward LLMs. From a total of 231 retrieved papers, we full-text reviewed 15 papers that recruited human evaluators to assess their experiences with LLMs. We report different biases and related concepts investigated by these studies, four broader LLM application areas, the evaluators' perceptions toward LLMs' performances including advantages, biases, and conflicting perceptions, factors influencing these perceptions, and concerns about LLM applications. △ Less

Submitted 2 March, 2024; v1 submitted 25 September, 2023; originally announced September 2023.

1.5 million materials narratives generated by chatbots

Authors: Yang Jeong Park, Sung Eun Jerng, Jin-Sung Park, Choah Kwon, Chia-Wei Hsu, Zhichu Ren, Sungroh Yoon, Ju Li

Abstract: The advent of artificial intelligence (AI) has enabled a comprehensive exploration of materials for various applications. However, AI models often prioritize frequently encountered materials in the scientific literature, limiting the selection of suitable candidates based on inherent physical and chemical properties. To address this imbalance, we have generated a dataset of 1,494,017 natural langu… ▽ More The advent of artificial intelligence (AI) has enabled a comprehensive exploration of materials for various applications. However, AI models often prioritize frequently encountered materials in the scientific literature, limiting the selection of suitable candidates based on inherent physical and chemical properties. To address this imbalance, we have generated a dataset of 1,494,017 natural language-material paragraphs based on combined OQMD, Materials Project, JARVIS, COD and AFLOW2 databases, which are dominated by ab initio calculations and tend to be much more evenly distributed on the periodic table. The generated text narratives were then polled and scored by both human experts and ChatGPT-4, based on three rubrics: technical accuracy, language and structure, and relevance and depth of content, showing similar scores but with human-scored depth of content being the most lagging. The merger of multi-modality data sources and large language model (LLM) holds immense potential for AI frameworks to help the exploration and discovery of solid-state materials for specific applications. △ Less

Submitted 25 August, 2023; originally announced August 2023.

A Hybrid Genetic Algorithm for the min-max Multiple Traveling Salesman Problem

Authors: Sasan Mahmoudinazlou, Changhyun Kwon

Abstract: This paper proposes a hybrid genetic algorithm for solving the Multiple Traveling Salesman Problem (mTSP) to minimize the length of the longest tour. The genetic algorithm utilizes a TSP sequence as the representation of each individual, and a dynamic programming algorithm is employed to evaluate the individual and find the optimal mTSP solution for the given sequence of cities. A novel crossover… ▽ More This paper proposes a hybrid genetic algorithm for solving the Multiple Traveling Salesman Problem (mTSP) to minimize the length of the longest tour. The genetic algorithm utilizes a TSP sequence as the representation of each individual, and a dynamic programming algorithm is employed to evaluate the individual and find the optimal mTSP solution for the given sequence of cities. A novel crossover operator is designed to combine similar tours from two parents and offers great diversity for the population. For some of the generated offspring, we detect and remove intersections between tours to obtain a solution with no intersections. This is particularly useful for the min-max mTSP. The generated offspring are also improved by a self-adaptive random local search and a thorough neighborhood search. Our algorithm outperforms all existing algorithms on average, with similar cutoff time thresholds, when tested against multiple benchmark sets found in the literature. Additionally, we improve the best-known solutions for $21$ out of $89$ instances on four benchmark sets. △ Less

Submitted 28 October, 2023; v1 submitted 13 July, 2023; originally announced July 2023.

A Neural Separation Algorithm for the Rounded Capacity Inequalities

Authors: Hyeonah Kim, Jinkyoo Park, Changhyun Kwon

Abstract: The cutting plane method is a key technique for successful branch-and-cut and branch-price-and-cut algorithms that find the exact optimal solutions for various vehicle routing problems (VRPs). Among various cuts, the rounded capacity inequalities (RCIs) are the most fundamental. To generate RCIs, we need to solve the separation problem, whose exact solution takes a long time to obtain; therefore,… ▽ More The cutting plane method is a key technique for successful branch-and-cut and branch-price-and-cut algorithms that find the exact optimal solutions for various vehicle routing problems (VRPs). Among various cuts, the rounded capacity inequalities (RCIs) are the most fundamental. To generate RCIs, we need to solve the separation problem, whose exact solution takes a long time to obtain; therefore, heuristic methods are widely used. We design a learning-based separation heuristic algorithm with graph coarsening that learns the solutions of the exact separation problem with a graph neural network (GNN), which is trained with small instances of 50 to 100 customers. We embed our separation algorithm within the cutting plane method to find a lower bound for the capacitated VRP (CVRP) with up to 1,000 customers. We compare the performance of our approach with CVRPSEP, a popular separation software package for various cuts used in solving VRPs. Our computational results show that our approach finds better lower bounds than CVRPSEP for large-scale problems with 400 or more customers, while CVRPSEP shows strong competency for problems with less than 400 customers. △ Less

Submitted 28 October, 2023; v1 submitted 29 June, 2023; originally announced June 2023.

RL4CO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark

Authors: Federico Berto, Chuanbo Hua, Junyoung Park, Laurin Luttmann, Yining Ma, Fanchen Bu, Jiarui Wang, Haoran Ye, Minsu Kim, Sanghyeok Choi, Nayeli Gast Zepeda, André Hottung, Jianan Zhou, Jieyi Bi, Yu Hu, Fei Liu, Hyeonah Kim, Jiwoo Son, Haeyeon Kim, Davide Angioni, Wouter Kool, Zhiguang Cao, Qingfu Zhang, Joungho Kim, Jie Zhang , et al. (8 additional authors not shown)

Abstract: Combinatorial optimization (CO) is fundamental to several real-world applications, from logistics and scheduling to hardware design and resource allocation. Deep reinforcement learning (RL) has recently shown significant benefits in solving CO problems, reducing reliance on domain expertise and improving computational efficiency. However, the absence of a unified benchmarking framework leads to in… ▽ More Combinatorial optimization (CO) is fundamental to several real-world applications, from logistics and scheduling to hardware design and resource allocation. Deep reinforcement learning (RL) has recently shown significant benefits in solving CO problems, reducing reliance on domain expertise and improving computational efficiency. However, the absence of a unified benchmarking framework leads to inconsistent evaluations, limits reproducibility, and increases engineering overhead, raising barriers to adoption for new researchers. To address these challenges, we introduce RL4CO, a unified and extensive benchmark with in-depth library coverage of 27 CO problem environments and 23 state-of-the-art baselines. Built on efficient software libraries and best practices in implementation, RL4CO features modularized implementation and flexible configurations of diverse environments, policy architectures, RL algorithms, and utilities with extensive documentation. RL4CO helps researchers build on existing successes while exploring and developing their own designs, facilitating the entire research process by decoupling science from heavy engineering. We finally provide extensive benchmark studies to inspire new insights and future work. RL4CO has already attracted numerous researchers in the community and is open-sourced at https://github.com/ai4co/rl4co. △ Less

Submitted 29 May, 2025; v1 submitted 29 June, 2023; originally announced June 2023.

Comments: KDD 2025 Oral

Towards Visualization Thumbnail Designs that Entice Reading Data-driven Articles

Authors: Hwiyeon Kim, Joohee Kim, Yunha Han, Hwajung Hong, Oh-Sang Kwon, Young-Woo Park, Niklas Elmqvist, Sungahn Ko, Bum Chul Kwon

Abstract: As online news increasingly include data journalism, there is a corresponding increase in the incorporation of visualization in article thumbnail images. However, little research exists on the design rationale for visualization thumbnails, such as resizing, cropping, simplifying, and embellishing charts that appear within the body of the associated article. Therefore, in this paper we aim to under… ▽ More As online news increasingly include data journalism, there is a corresponding increase in the incorporation of visualization in article thumbnail images. However, little research exists on the design rationale for visualization thumbnails, such as resizing, cropping, simplifying, and embellishing charts that appear within the body of the associated article. Therefore, in this paper we aim to understand these design choices and determine what makes a visualization thumbnail inviting and interpretable. To this end, we first survey visualization thumbnails collected online and discuss visualization thumbnail practices with data journalists and news graphics designers. Based on the survey and discussion results, we then define a design space for visualization thumbnails and conduct a user study with four types of visualization thumbnails derived from the design space. The study results indicate that different chart components play different roles in attracting reader attention and enhancing reader understandability of the visualization thumbnails. We also find various thumbnail design strategies for effectively combining the charts' components, such as a data summary with highlights and data labels, and a visual legend with text labels and Human Recognizable Objects (HROs), into thumbnails. Ultimately, we distill our findings into design implications that allow effective visualization thumbnail designs for data-rich news articles. Our work can thus be seen as a first step toward providing structured guidance on how to design compelling thumbnails for data stories. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Comments: To appear in IEEE Transactions on Visualization and Computer Graphics, 16 pages, 6 figures, 5 tables. arXiv admin note: text overlap with arXiv:1908.06922

Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models

Authors: Bum Chul Kwon, Nandana Mihindukulasooriya

Abstract: Pre-trained transformer-based language models are becoming increasingly popular due to their exceptional performance on various benchmarks. However, concerns persist regarding the presence of hidden biases within these models, which can lead to discriminatory outcomes and reinforce harmful stereotypes. To address this issue, we propose Finspector, a human-centered visual inspection tool designed t… ▽ More Pre-trained transformer-based language models are becoming increasingly popular due to their exceptional performance on various benchmarks. However, concerns persist regarding the presence of hidden biases within these models, which can lead to discriminatory outcomes and reinforce harmful stereotypes. To address this issue, we propose Finspector, a human-centered visual inspection tool designed to detect biases in different categories through log-likelihood scores generated by language models. The goal of the tool is to enable researchers to easily identify potential biases using visual analytics, ultimately contributing to a fairer and more just deployment of these models in both academic and industrial settings. Finspector is available at https://github.com/IBM/finspector. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Comments: ACL 2023 System Demonstrations, 9 pages, 3 figures

PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models

Authors: Aditi Mishra, Utkarsh Soni, Anjana Arunkumar, Jinbin Huang, Bum Chul Kwon, Chris Bryan

Abstract: Large Language Models (LLMs) have gained widespread popularity due to their ability to perform ad-hoc Natural Language Processing (NLP) tasks with a simple natural language prompt. Part of the appeal for LLMs is their approachability to the general public, including individuals with no prior technical experience in NLP techniques. However, natural language prompts can vary significantly in terms o… ▽ More Large Language Models (LLMs) have gained widespread popularity due to their ability to perform ad-hoc Natural Language Processing (NLP) tasks with a simple natural language prompt. Part of the appeal for LLMs is their approachability to the general public, including individuals with no prior technical experience in NLP techniques. However, natural language prompts can vary significantly in terms of their linguistic structure, context, and other semantics. Modifying one or more of these aspects can result in significant differences in task performance. Non-expert users may find it challenging to identify the changes needed to improve a prompt, especially when they lack domain-specific knowledge and lack appropriate feedback. To address this challenge, we present PromptAid, a visual analytics system designed to interactively create, refine, and test prompts through exploration, perturbation, testing, and iteration. PromptAid uses multiple, coordinated visualizations which allow users to improve prompts by using the three strategies: keyword perturbations, paraphrasing perturbations, and obtaining the best set of in-context few-shot examples. PromptAid was designed through an iterative prototyping process involving NLP experts and was evaluated through quantitative and qualitative assessments for LLMs. Our findings indicate that PromptAid helps users to iterate over prompt template alterations with less cognitive overhead, generate diverse prompts with help of recommendations, and analyze the performance of the generated prompts while surpassing existing state-of-the-art prompting interfaces in performance. △ Less

Submitted 22 February, 2025; v1 submitted 4 April, 2023; originally announced April 2023.

Causalvis: Visualizations for Causal Inference

Authors: Grace Guo, Ehud Karavani, Alex Endert, Bum Chul Kwon

Abstract: Causal inference is a statistical paradigm for quantifying causal effects using observational data. It is a complex process, requiring multiple steps, iterations, and collaborations with domain experts. Analysts often rely on visualizations to evaluate the accuracy of each step. However, existing visualization toolkits are not designed to support the entire causal inference process within computat… ▽ More Causal inference is a statistical paradigm for quantifying causal effects using observational data. It is a complex process, requiring multiple steps, iterations, and collaborations with domain experts. Analysts often rely on visualizations to evaluate the accuracy of each step. However, existing visualization toolkits are not designed to support the entire causal inference process within computational environments familiar to analysts. In this paper, we address this gap with Causalvis, a Python visualization package for causal inference. Working closely with causal inference experts, we adopted an iterative design process to develop four interactive visualization modules to support causal inference analysis tasks. The modules are then presented back to the experts for feedback and evaluation. We found that Causalvis effectively supported the iterative causal inference process. We discuss the implications of our findings for designing visualizations for causal inference, particularly for tasks of communication and collaboration. △ Less

Submitted 1 March, 2023; originally announced March 2023.

Comments: 20 pages, 14 figures

A Hybrid Genetic Algorithm with Type-Aware Chromosomes for Traveling Salesman Problems with Drone

Authors: Sasan Mahmoudinazlou, Changhyun Kwon

Abstract: There are emerging transportation problems known as the Traveling Salesman Problem with Drone (TSPD) and the Flying Sidekick Traveling Salesman Problem (FSTSP) that involve using a drone in conjunction with a truck for package delivery. This study presents a hybrid genetic algorithm for solving TSPD and FSTSP by incorporating local search and dynamic programming. Similar algorithms exist in the li… ▽ More There are emerging transportation problems known as the Traveling Salesman Problem with Drone (TSPD) and the Flying Sidekick Traveling Salesman Problem (FSTSP) that involve using a drone in conjunction with a truck for package delivery. This study presents a hybrid genetic algorithm for solving TSPD and FSTSP by incorporating local search and dynamic programming. Similar algorithms exist in the literature. Our algorithm, however, considers more sophisticated chromosomes and less computationally complex dynamic programming to enable broader exploration by the genetic algorithm and efficient exploitation through dynamic programming and local search. The key contribution of this paper is the discovery of how decision-making processes for solving TSPD and FSTSP should be divided among the layers of genetic algorithm, dynamic programming, and local search. In particular, our genetic algorithm generates the truck and the drone sequences separately and encodes them in a type-aware chromosome, wherein each customer is assigned to either the truck or the drone. We apply local search to each chromosome, which is decoded by dynamic programming for fitness evaluation. Our new algorithm is shown to outperform existing algorithms on most benchmark instances in both quality and time. Our algorithms found the new best solutions for 538 TSPD instances out of 920 and 74 FSTSP instances out of 132. △ Less

Submitted 29 April, 2024; v1 submitted 1 March, 2023; originally announced March 2023.

Learning-based Uncertainty-aware Navigation in 3D Off-Road Terrains

Authors: Hojin Lee, Junsung Kwon, Cheolhyeon Kwon

Abstract: This paper presents a safe, efficient, and agile ground vehicle navigation algorithm for 3D off-road terrain environments. Off-road navigation is subject to uncertain vehicle-terrain interactions caused by different terrain conditions on top of 3D terrain topology. The existing works are limited to adopt overly simplified vehicle-terrain models. The proposed algorithm learns the terrain-induced un… ▽ More This paper presents a safe, efficient, and agile ground vehicle navigation algorithm for 3D off-road terrain environments. Off-road navigation is subject to uncertain vehicle-terrain interactions caused by different terrain conditions on top of 3D terrain topology. The existing works are limited to adopt overly simplified vehicle-terrain models. The proposed algorithm learns the terrain-induced uncertainties from driving data and encodes the learned uncertainty distribution into the traversability cost for path evaluation. The navigation path is then designed to optimize the uncertainty-aware traversability cost, resulting in a safe and agile vehicle maneuver. Assuring real-time execution, the algorithm is further implemented within parallel computation architecture running on Graphics Processing Units (GPU). △ Less

Submitted 19 September, 2022; originally announced September 2022.

Comments: 6 pages, 6 figures, submitted to International Conference on Robotics and Automation (ICRA 2023)

RMExplorer: A Visual Analytics Approach to Explore the Performance and the Fairness of Disease Risk Models on Population Subgroups

Authors: Bum Chul Kwon, Uri Kartoun, Shaan Khurshid, Mikhail Yurochkin, Subha Maity, Deanna G Brockman, Amit V Khera, Patrick T Ellinor, Steven A Lubitz, Kenney Ng

Abstract: Disease risk models can identify high-risk patients and help clinicians provide more personalized care. However, risk models developed on one dataset may not generalize across diverse subpopulations of patients in different datasets and may have unexpected performance. It is challenging for clinical researchers to inspect risk models across different subgroups without any tools. Therefore, we deve… ▽ More Disease risk models can identify high-risk patients and help clinicians provide more personalized care. However, risk models developed on one dataset may not generalize across diverse subpopulations of patients in different datasets and may have unexpected performance. It is challenging for clinical researchers to inspect risk models across different subgroups without any tools. Therefore, we developed an interactive visualization system called RMExplorer (Risk Model Explorer) to enable interactive risk model assessment. Specifically, the system allows users to define subgroups of patients by selecting clinical, demographic, or other characteristics, to explore the performance and fairness of risk models on the subgroups, and to understand the feature contributions to risk scores. To demonstrate the usefulness of the tool, we conduct a case study, where we use RMExplorer to explore three atrial fibrillation risk models by applying them to the UK Biobank dataset of 445,329 individuals. RMExplorer can help researchers to evaluate the performance and biases of risk models on subpopulations of interest in their data. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: IEEE VIS 2022 Short

DASH: Visual Analytics for Debiasing Image Classification via User-Driven Synthetic Data Augmentation

Authors: Bum Chul Kwon, Jungsoo Lee, Chaeyeon Chung, Nyoungwoo Lee, Ho-Jin Choi, Jaegul Choo

Abstract: Image classification models often learn to predict a class based on irrelevant co-occurrences between input features and an output class in training data. We call the unwanted correlations "data biases," and the visual features causing data biases "bias factors." It is challenging to identify and mitigate biases automatically without human intervention. Therefore, we conducted a design study to fi… ▽ More Image classification models often learn to predict a class based on irrelevant co-occurrences between input features and an output class in training data. We call the unwanted correlations "data biases," and the visual features causing data biases "bias factors." It is challenging to identify and mitigate biases automatically without human intervention. Therefore, we conducted a design study to find a human-in-the-loop solution. First, we identified user tasks that capture the bias mitigation process for image classification models with three experts. Then, to support the tasks, we developed a visual analytics system called DASH that allows users to visually identify bias factors, to iteratively generate synthetic images using a state-of-the-art image-to-image translation model, and to supervise the model training process for improving the classification accuracy. Our quantitative evaluation and qualitative study with ten participants demonstrate the usefulness of DASH and provide lessons for future work. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: 5 pages, 3 figures, EuroVis 2022 Short, Honorable Mention

Survey on the State-of-the-Art in Device-to-Device Communication: A Resource Allocation Perspective

Authors: Tariq Islam, Cheolhyeon Kwon

Abstract: Device to Device (D2D) communication takes advantage of the proximity between the communicating devices in order to achieve efficient resource utilization, improved throughput and energy efficiency, simultaneous serviceability and reduced latency. One of the main characteristics of D2D communication is reuse of the frequency resource in order to improve spectral efficiency of the system. Neverthel… ▽ More Device to Device (D2D) communication takes advantage of the proximity between the communicating devices in order to achieve efficient resource utilization, improved throughput and energy efficiency, simultaneous serviceability and reduced latency. One of the main characteristics of D2D communication is reuse of the frequency resource in order to improve spectral efficiency of the system. Nevertheless, frequency reuse introduces significantly high interference levels thus necessitating efficient resource allocation algorithms that can enable simultaneous communication sessions through effective channel and/or power allocation. This survey paper presents a comprehensive investigation of the state-of-the-art resource allocation algorithms in D2D communication underlaying cellular networks. The surveyed algorithms are evaluated based on heterogeneous parameters which constitute the elementary features of a resource allocation algorithm in D2D paradigm. Additionally, in order to familiarize the readers with the basic design of the surveyed resource allocation algorithms, brief description of the mode of operation of each algorithm is presented. The surveyed algorithms are divided into four categories based on their technical doctrine i.e., conventional optimization based, Non-Orthogonal-Multiple-Access (NOMA) based, game theory based and machine learning based techniques. Towards the end, several open challenges are remarked as the future research directions in resource allocation for D2D communication. △ Less

Submitted 19 May, 2022; originally announced June 2022.

Comments: 29 pages, 5 figures,

Statistical inference as Green's functions

Authors: Hyun Keun Lee, Chulan Kwon, Yong Woon Kim

Abstract: Statistical inference from data is a foundational task in science. Recently, it has received growing attention for its central role in inference systems of primary interest in data sciences and machine learning. However, the understanding of statistical inference is not that solid while remains as a matter of subjective belief or as the routine procedures once claimed objective. We here show that… ▽ More Statistical inference from data is a foundational task in science. Recently, it has received growing attention for its central role in inference systems of primary interest in data sciences and machine learning. However, the understanding of statistical inference is not that solid while remains as a matter of subjective belief or as the routine procedures once claimed objective. We here show that there is an objective description of statistical inference for long sequence of exchangeable binary random variables, the prototypal stochasticity in theories and applications. A linear differential equation is derived from the identity known as de Finetti's representation theorem, and it turns out that statistical inference is given by the Green's functions. Our finding is an answer to the normative issue of science that pursues the objectivity based on data, and its significance will be far-reaching in most pure and applied fields. △ Less

Submitted 10 October, 2022; v1 submitted 23 May, 2022; originally announced May 2022.

Comments: 21 pages, 1 figure

An Empirical Study on the Relationship Between the Number of Coordinated Views and Visual Analysis

Authors: Juyoung Oh, Chunggi Lee, Hwiyeon Kim, Kihwan Kim, Osang Kwon, Eric D. Ragan, Bum Chul Kwon, Sungahn Ko

Abstract: Coordinated Multiple views (CMVs) are a visualization technique that simultaneously presents multiple visualizations in separate but linked views. There are many studies that report the advantages (e.g., usefulness for finding hidden relationships) and disadvantages (e.g., cognitive load) of CMVs. But little empirical work exists on the impact of the number of views on visual anlaysis results and… ▽ More Coordinated Multiple views (CMVs) are a visualization technique that simultaneously presents multiple visualizations in separate but linked views. There are many studies that report the advantages (e.g., usefulness for finding hidden relationships) and disadvantages (e.g., cognitive load) of CMVs. But little empirical work exists on the impact of the number of views on visual anlaysis results and processes, which results in uncertainty in the relationship between the view number and visual anlaysis. In this work, we aim at investigating the relationship between the number of coordinated views and users analytic processes and results. To achieve the goal, we implemented a CMV tool for visual anlaysis. We also provided visualization duplication in the tool to help users easily create a desired number of visualization views on-the-fly. We conducted a between-subject study with 44 participants, where we asked participants to solve five analytic problems using the visual tool. Through quantitative and qualitative analysis, we discovered the positive correlation between the number of views and analytic results. We also found that visualization duplication encourages users to create more views and to take various analysis strategies. Based on the results, we provide implications and limitations of our study. △ Less

Submitted 20 April, 2022; originally announced April 2022.

ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspective

Authors: Jinbin Huang, Aditi Mishra, Bum Chul Kwon, Chris Bryan

Abstract: Traditional deep learning interpretability methods which are suitable for model users cannot explain network behaviors at the global level and are inflexible at providing fine-grained explanations. As a solution, concept-based explanations are gaining attention due to their human intuitiveness and their flexibility to describe both global and local model behaviors. Concepts are groups of similarly… ▽ More Traditional deep learning interpretability methods which are suitable for model users cannot explain network behaviors at the global level and are inflexible at providing fine-grained explanations. As a solution, concept-based explanations are gaining attention due to their human intuitiveness and their flexibility to describe both global and local model behaviors. Concepts are groups of similarly meaningful pixels that express a notion, embedded within the network's latent space and have commonly been hand-generated, but have recently been discovered by automated approaches. Unfortunately, the magnitude and diversity of discovered concepts makes it difficult to navigate and make sense of the concept space. Visual analytics can serve a valuable role in bridging these gaps by enabling structured navigation and exploration of the concept space to provide concept-based insights of model behavior to users. To this end, we design, develop, and validate ConceptExplainer, a visual analytics system that enables people to interactively probe and explore the concept space to explain model behavior at the instance/class/global level. The system was developed via iterative prototyping to address a number of design challenges that model users face in interpreting the behavior of deep learning models. Via a rigorous user study, we validate how ConceptExplainer supports these challenges. Likewise, we conduct a series of usage scenarios to demonstrate how the system supports the interactive analysis of model behavior across a variety of tasks and explanation granularities, such as identifying concepts that are important to classification, identifying bias in training data, and understanding how concepts can be shared across diverse and seemingly dissimilar classes. △ Less

Submitted 24 October, 2022; v1 submitted 4 April, 2022; originally announced April 2022.

Comments: 9 pages, 6 figures

A Deep Reinforcement Learning Approach for Solving the Traveling Salesman Problem with Drone

Authors: Aigerim Bogyrbayeva, Taehyun Yoon, Hanbum Ko, Sungbin Lim, Hyokun Yun, Changhyun Kwon

Abstract: Reinforcement learning has recently shown promise in learning quality solutions in many combinatorial optimization problems. In particular, the attention-based encoder-decoder models show high effectiveness on various routing problems, including the Traveling Salesman Problem (TSP). Unfortunately, they perform poorly for the TSP with Drone (TSP-D), requiring routing a heterogeneous fleet of vehicl… ▽ More Reinforcement learning has recently shown promise in learning quality solutions in many combinatorial optimization problems. In particular, the attention-based encoder-decoder models show high effectiveness on various routing problems, including the Traveling Salesman Problem (TSP). Unfortunately, they perform poorly for the TSP with Drone (TSP-D), requiring routing a heterogeneous fleet of vehicles in coordination -- a truck and a drone. In TSP-D, the two vehicles are moving in tandem and may need to wait at a node for the other vehicle to join. State-less attention-based decoder fails to make such coordination between vehicles. We propose a hybrid model that uses an attention encoder and a Long Short-Term Memory (LSTM) network decoder, in which the decoder's hidden state can represent the sequence of actions made. We empirically demonstrate that such a hybrid model improves upon a purely attention-based model for both solution quality and computational efficiency. Our experiments on the min-max Capacitated Vehicle Routing Problem (mmCVRP) also confirm that the hybrid model is more suitable for the coordinated routing of multiple vehicles than the attention-based model. The proposed model demonstrates comparable results as the operations research baseline methods. △ Less

Submitted 5 December, 2022; v1 submitted 21 December, 2021; originally announced December 2021.

Modeling Disease Progression Trajectories from Longitudinal Observational Data

Authors: Bum Chul Kwon, Peter Achenbach, Jessica L. Dunne, William Hagopian, Markus Lundgren, Kenney Ng, Riitta Veijola, Brigitte I. Frohnert, Vibha Anand, the T1DI Study Group

Abstract: Analyzing disease progression patterns can provide useful insights into the disease processes of many chronic conditions. These analyses may help inform recruitment for prevention trials or the development and personalization of treatments for those affected. We learn disease progression patterns using Hidden Markov Models (HMM) and distill them into distinct trajectories using visualization metho… ▽ More Analyzing disease progression patterns can provide useful insights into the disease processes of many chronic conditions. These analyses may help inform recruitment for prevention trials or the development and personalization of treatments for those affected. We learn disease progression patterns using Hidden Markov Models (HMM) and distill them into distinct trajectories using visualization methods. We apply it to the domain of Type 1 Diabetes (T1D) using large longitudinal observational data from the T1DI study group. Our method discovers distinct disease progression trajectories that corroborate with recently published findings. In this paper, we describe the iterative process of developing the model. These methods may also be applied to other chronic conditions that evolve over time. △ Less

Submitted 9 December, 2020; originally announced December 2020.

Comments: 9 pages, 5 figures, to be published in proceedings of AMIA Annual Symposium 2020

A Reinforcement Learning Approach for Rebalancing Electric Vehicle Sharing Systems

Authors: Aigerim Bogyrbayeva, Sungwook Jang, Ankit Shah, Young Jae Jang, Changhyun Kwon

Abstract: This paper proposes a reinforcement learning approach for nightly offline rebalancing operations in free-floating electric vehicle sharing systems (FFEVSS). Due to sparse demand in a network, FFEVSS require relocation of electrical vehicles (EVs) to charging stations and demander nodes, which is typically done by a group of drivers. A shuttle is used to pick up and drop off drivers throughout the… ▽ More This paper proposes a reinforcement learning approach for nightly offline rebalancing operations in free-floating electric vehicle sharing systems (FFEVSS). Due to sparse demand in a network, FFEVSS require relocation of electrical vehicles (EVs) to charging stations and demander nodes, which is typically done by a group of drivers. A shuttle is used to pick up and drop off drivers throughout the network. The objective of this study is to solve the shuttle routing problem to finish the rebalancing work in the minimal time. We consider a reinforcement learning framework for the problem, in which a central controller determines the routing policies of a fleet of multiple shuttles. We deploy a policy gradient method for training recurrent neural networks and compare the obtained policy results with heuristic solutions. Our numerical studies show that unlike the existing solutions in the literature, the proposed methods allow to solve the general version of the problem with no restrictions on the urban EV network structure and charging requirements of EVs. Moreover, the learned policies offer a wide range of flexibility resulting in a significant reduction in the time needed to rebalance the network. △ Less

Submitted 6 April, 2021; v1 submitted 5 October, 2020; originally announced October 2020.

User-driven Analysis of Longitudinal Health Data with Hidden Markov Models for Clinical Insights

Authors: Bum Chul Kwon

Abstract: A goal of clinical researchers is to understand the progression of a disease through a set of biomarkers. Researchers often conduct observational studies, where they collect numerous samples from selected subjects throughout multiple years. Hidden Markov Models (HMMs) can be applied to discover latent states and their transition probabilities over time. However, it is challenging for clinical rese… ▽ More A goal of clinical researchers is to understand the progression of a disease through a set of biomarkers. Researchers often conduct observational studies, where they collect numerous samples from selected subjects throughout multiple years. Hidden Markov Models (HMMs) can be applied to discover latent states and their transition probabilities over time. However, it is challenging for clinical researchers to interpret the outcomes and to gain insights about the disease. Thus, this demo introduces an interactive visualization system called DPVis, which was designed to help researchers to interactively explore HMM outcomes. The demo provides guidelines of how to implement the clinician-in-the-loop approach for analyzing longitudinal, observational health data with visual analytics. △ Less

Submitted 24 July, 2020; originally announced July 2020.

Comments: Demo presented in KDD 2020 Workshop on Applied Data Science for Healthcare

Sensitivity of Wardrop Equilibria: Revisited

Authors: Mahdi Takalloo, Changhyun Kwon

Abstract: For single-commodity networks, the increase of the price of anarchy is bounded by a factor of $(1+ε)^p$ from above, when the travel demand is increased by a factor of $1+ε$ and the latency functions are polynomials of degree at most $p$. We show that the same upper bound holds for multi-commodity networks and provide a lower bound as well. For single-commodity networks, the increase of the price of anarchy is bounded by a factor of $(1+ε)^p$ from above, when the travel demand is increased by a factor of $1+ε$ and the latency functions are polynomials of degree at most $p$. We show that the same upper bound holds for multi-commodity networks and provide a lower bound as well. △ Less

Submitted 13 February, 2020; originally announced February 2020.

GUIComp: A GUI Design Assistant with Real-Time, Multi-Faceted Feedback

Authors: Chunggi Lee, Sanghoon Kim, Dongyun Han, Hongjun Yang, Young-Woo Park, Bum Chul Kwon, Sungahn Ko

Abstract: Users may face challenges while designing graphical user interfaces, due to a lack of relevant experience and guidance. This paper aims to investigate the issues that users with no experience face during the design process, and how to resolve them. To this end, we conducted semi-structured interviews, based on which we built a GUI prototyping assistance tool called GUIComp. This tool can be connec… ▽ More Users may face challenges while designing graphical user interfaces, due to a lack of relevant experience and guidance. This paper aims to investigate the issues that users with no experience face during the design process, and how to resolve them. To this end, we conducted semi-structured interviews, based on which we built a GUI prototyping assistance tool called GUIComp. This tool can be connected to GUI design software as an extension, and it provides real-time, multi-faceted feedback on a user's current design. Additionally, we conducted two user studies, in which we asked participants to create mobile GUIs with or without GUIComp, and requested online workers to assess the created GUIs. The experimental results show that GUIComp facilitated iterative design and the participants with GUIComp had better a user experience and produced more acceptable designs than those who did not. △ Less

Submitted 16 January, 2020; originally announced January 2020.

Comments: 10 pages, 3 figures, accepted paper to CHI 2020 conference

On the Price of Satisficing in Network User Equilibria

Authors: Mahdi Takalloo, Changhyun Kwon

Abstract: When network users are satisficing decision-makers, the resulting traffic pattern attains a satisficing user equilibrium, which may deviate from the (perfectly rational) user equilibrium. In a satisficing user equilibrium traffic pattern, the total system travel time can be worse than in the case of the PRUE. We show how bad the worst-case satisficing user equilibrium traffic pattern can be, compa… ▽ More When network users are satisficing decision-makers, the resulting traffic pattern attains a satisficing user equilibrium, which may deviate from the (perfectly rational) user equilibrium. In a satisficing user equilibrium traffic pattern, the total system travel time can be worse than in the case of the PRUE. We show how bad the worst-case satisficing user equilibrium traffic pattern can be, compared to the perfectly rational user equilibrium. We call the ratio between the total system travel times of the two traffic patterns the price of satisficing, for which we provide an analytical bound. We compare the analytical bound with numerical bounds for several transportation networks. △ Less

Submitted 18 November, 2019; originally announced November 2019.

Geono-Cluster: Interactive Visual Cluster Analysis for Biologists

Authors: Bahador Saket, Subhajit Das, Bum Chul Kwon, Alex Endert

Abstract: Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grouping in their data using existing tools. Visual cluster analysis is currently performed either programmatically or throug… ▽ More Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grouping in their data using existing tools. Visual cluster analysis is currently performed either programmatically or through menus and dialogues in many tools, which require parameter adjustments over several steps of trial-and-error. In this paper, we introduce Geono-Cluster, a novel visual analysis tool designed to support cluster analysis for biologists who do not have formal data science training. Geono-Cluster enables biologists to apply their domain expertise into clustering results by visually demonstrating how their expected clustering outputs should look like with a small sample of data instances. The system then predicts users' intentions and generates potential clustering results. Our study follows the design study protocol to derive biologists' tasks and requirements, design the system, and evaluate the system with experts on their own dataset. Results of our study with six biologists provide initial evidence that Geono-Cluster enables biologists to create, refine, and evaluate clustering results to effectively analyze their data and gain data-driven insights. At the end, we discuss lessons learned and the implications of our study. △ Less

Submitted 3 November, 2019; originally announced November 2019.

SANVis: Visual Analytics for Understanding Self-Attention Networks

Authors: Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jaehyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, Jaegul Choo

Abstract: Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set o… ▽ More Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set of vectors. Such encoding aims at simultaneously capturing diverse syntactic and semantic features within a set, each of which corresponds to a particular attention head, forming altogether multi-head attention. Meanwhile, the increased model complexity prevents users from easily understanding and manipulating the inner workings of models. To tackle the challenges, we present a visual analytics system called SANVis, which helps users understand the behaviors and the characteristics of multi-head self-attention networks. Using a state-of-the-art self-attention model called Transformer, we demonstrate usage scenarios of SANVis in machine translation tasks. Our system is available at http://short.sanvis.org △ Less

Submitted 13 September, 2019; originally announced September 2019.

Comments: VAST Short - IEEE VIS 2019

Thumbnails for Data Stories: A Survey of Current Practices

Authors: Hwiyeon Kim, Juyoung Oh, Yunha Han, Sungahn Ko, Matthew Brehmer, Bum Chul Kwon

Abstract: When people browse online news, small thumbnail images accompanying links to articles attract their attention and help them to decide which articles to read. As an increasing proportion of online news can be construed as data journalism, we have witnessed a corresponding increase in the incorporation of visualization in article thumbnails. However, there is little research to support alternative d… ▽ More When people browse online news, small thumbnail images accompanying links to articles attract their attention and help them to decide which articles to read. As an increasing proportion of online news can be construed as data journalism, we have witnessed a corresponding increase in the incorporation of visualization in article thumbnails. However, there is little research to support alternative design choices for visualization thumbnails, which include resizing, cropping, simplifying, and embellishing charts appearing within the body of the associated article. We therefore sought to better understand these design choices and determine what makes a visualization thumbnail inviting and interpretable. This paper presents our findings from a survey of visualization thumbnails collected online and from conversations with data journalists and news graphics designers. Our study reveals that there exists an uncharted design space, one that is in need of further empirical study. Our work can thus be seen as a first step toward providing structured guidance on how to design thumbnails for data stories. △ Less

Submitted 19 August, 2019; originally announced August 2019.

Comments: To appear at IEEE VIS 2019

DPVis: Visual Analytics with Hidden Markov Models for Disease Progression Pathways

Authors: Bum Chul Kwon, Vibha Anand, Kristen A Severson, Soumya Ghosh, Zhaonan Sun, Brigitte I Frohnert, Markus Lundgren, Kenney Ng

Abstract: Clinical researchers use disease progression models to understand patient status and characterize progression patterns from longitudinal health records. One approach for disease progression modeling is to describe patient status using a small number of states that represent distinctive distributions over a set of observed measures. Hidden Markov models (HMMs) and its variants are a class of models… ▽ More Clinical researchers use disease progression models to understand patient status and characterize progression patterns from longitudinal health records. One approach for disease progression modeling is to describe patient status using a small number of states that represent distinctive distributions over a set of observed measures. Hidden Markov models (HMMs) and its variants are a class of models that both discover these states and make inferences of health states for patients. Despite the advantages of using the algorithms for discovering interesting patterns, it still remains challenging for medical experts to interpret model outputs, understand complex modeling parameters, and clinically make sense of the patterns. To tackle these problems, we conducted a design study with clinical scientists, statisticians, and visualization experts, with the goal to investigate disease progression pathways of chronic diseases, namely type 1 diabetes (T1D), Huntington's disease, Parkinson's disease, and chronic obstructive pulmonary disease (COPD). As a result, we introduce DPVis which seamlessly integrates model parameters and outcomes of HMMs into interpretable and interactive visualizations. In this study, we demonstrate that DPVis is successful in evaluating disease progression models, visually summarizing disease states, interactively exploring disease progression patterns, and building, analyzing, and comparing clinically relevant patient subgroups. △ Less

Submitted 9 April, 2020; v1 submitted 25 April, 2019; originally announced April 2019.

Comments: to appear at IEEE Transactions on Visualization and Computer Graphics

RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records

Authors: Bum Chul Kwon, Min-Je Choi, Joanne Taery Kim, Edward Choi, Young Bin Kim, Soonwook Kwon, Jimeng Sun, Jaegul Choo

Abstract: We have recently seen many successful applications of recurrent neural networks (RNNs) on electronic medical records (EMRs), which contain histories of patients' diagnoses, medications, and other various events, in order to predict the current and future states of patients. Despite the strong performance of RNNs, it is often challenging for users to understand why the model makes a particular pred… ▽ More We have recently seen many successful applications of recurrent neural networks (RNNs) on electronic medical records (EMRs), which contain histories of patients' diagnoses, medications, and other various events, in order to predict the current and future states of patients. Despite the strong performance of RNNs, it is often challenging for users to understand why the model makes a particular prediction. Such black-box nature of RNNs can impede its wide adoption in clinical practice. Furthermore, we have no established methods to interactively leverage users' domain expertise and prior knowledge as inputs for steering the model. Therefore, our design study aims to provide a visual analytics solution to increase interpretability and interactivity of RNNs via a joint effort of medical experts, artificial intelligence scientists, and visual analytics researchers. Following the iterative design process between the experts, we design, implement, and evaluate a visual analytics tool called RetainVis, which couples a newly improved, interpretable and interactive RNN-based model called RetainEX and visualizations for users' exploration of EMR data in the context of prediction tasks. Our study shows the effective use of RetainVis for gaining insights into how individual medical codes contribute to making risk predictions, using EMRs of patients with heart failure and cataract symptoms. Our study also demonstrates how we made substantial changes to the state-of-the-art RNN model called RETAIN in order to make use of temporal information and increase interactivity. This study will provide a useful guideline for researchers that aim to design an interpretable and interactive visual analytics tool for RNNs. △ Less

Submitted 23 October, 2018; v1 submitted 27 May, 2018; originally announced May 2018.

Comments: Accepted at IEEE VIS 2018. To appear in IEEE Transactions on Visualization and Computer Graphics in January 2019

Coddlers, Scientists, Adventurers, and Opportunists: Personas to Inform Online Health Community Development

Authors: Jina Huh, Bum Chul Kwon, Jaegul Choo, Sung-Hee Kim, Ji Soo Yi

Abstract: As online health communities (OHCs) grow, users find it challenging to properly search, read, and contribute to the community because of its overwhelming content. Our goal is to understand OHC users' needs and requirements for better delivering large-scale OHC content. We interviewed 14 OHC users with interests in diabetes to investigate their attitudes and needs towards using OHCs and 2 OHC admin… ▽ More As online health communities (OHCs) grow, users find it challenging to properly search, read, and contribute to the community because of its overwhelming content. Our goal is to understand OHC users' needs and requirements for better delivering large-scale OHC content. We interviewed 14 OHC users with interests in diabetes to investigate their attitudes and needs towards using OHCs and 2 OHC administrators to assess our findings. Four personas -Coddlers, Scientists, Adventurers, and Opportunists- emerged, which inform users' interaction behavior and attitudes with OHCs. An individual can possess the characteristics of multiple personas, which can also change over time. Our personas uniquely describe users' OHC participation intertwined with illness contexts compared to existing social types in general online communities. We discuss broader implications back to the literature and how our findings apply to other illness contexts in OHCs. We end with requirements for personalized delivery of large-scale OHC content. △ Less

Submitted 30 June, 2015; originally announced June 2015.

Comments: 10 pages, 1 table

Competitive Robust Dynamic Pricing in Continuous Time with Fixed Inventories

Authors: Terry L. Friesz, Changhyun Kwon, Tae Il Kim, Lifan Fan, Tao Yao

Abstract: The problem of robust dynamic pricing of an abstract commodity, whose inventory is specified at an initial time but never subsequently replenished, originally studied by Perakis and Sood (2006) in discrete time, is considered from the perspective of continuous time. We use a multiplicative demand function to model the uncertain demand, and develop a robust counterpart to replace the uncertain dema… ▽ More The problem of robust dynamic pricing of an abstract commodity, whose inventory is specified at an initial time but never subsequently replenished, originally studied by Perakis and Sood (2006) in discrete time, is considered from the perspective of continuous time. We use a multiplicative demand function to model the uncertain demand, and develop a robust counterpart to replace the uncertain demand constraint. The sellers' robust best response problem yields a generalized Nash equilibrium problem, which can be formulated as an equivalent, continuous-time quasi-variational inequality. We demonstrate that, for appropriate regularity conditions, a generalized robust Nash equilibrium exists. We show that the quasi-variational inequality may be replaced by an equivalent variational inequality, and use a fixed-point algorithm to solve the variational inequality. We also demonstrate how explicit time lags associated with price updating in real-world decision environments, as well as specific pricing decision rules, may be introduced to create a dual time scale formulation and the associated solutions computed. We illustrate, via numerical examples, how robust pricing based on our DPFI formulation offers generally superior and never inferior worst case performance compared to nominal pricing. △ Less

Submitted 21 August, 2012; originally announced August 2012.

On the existence of potential landscape in the evolution of complex systems

Authors: P. Ao, C. Kwon, H. Qian

Abstract: A recently developed treatment of stochastic processes leads to the construction of a potential landscape for the dynamical evolution of complex systems. Since the existence of a potential function in generic settings has been frequently questioned in literature,herewe study several related theoretical issues that lie at core of the construction. We showthat the novel treatment,via a transformat… ▽ More A recently developed treatment of stochastic processes leads to the construction of a potential landscape for the dynamical evolution of complex systems. Since the existence of a potential function in generic settings has been frequently questioned in literature,herewe study several related theoretical issues that lie at core of the construction. We showthat the novel treatment,via a transformation,is closely related to the symplectic structure that is central in many branches of theoretical physics. Using this insight, we demonstrate an invariant under the transformation. We further explicitly demonstrate, in one-dimensional case, the contradistinction among the new treatment to those of Ito and Stratonovich, as well as others.Our results strongly suggest that the method from statistical physics can be useful in studying stochastic, complex systems in general. △ Less

Submitted 20 March, 2007; originally announced March 2007.

Comments: latex, 18 pages

Journal ref: Complexity 12 (2007) 19-27