Showing 1–9 of 9 results for author: Chapman, W

Investigating the contribution of terrain-following coordinates and conservation schemes in AI-driven precipitation forecasts

Authors: Yingkai Sha, John S. Schreck, William Chapman, David John Gagne II

Abstract: Artificial Intelligence (AI) weather prediction (AIWP) models often produce "blurry" precipitation forecasts that overestimate drizzle and underestimate extremes. This study provides a novel solution to tackle this problem -- integrating terrain-following coordinates with global mass and energy conservation schemes into AIWP models. Forecast experiments are conducted to evaluate the effectiveness… ▽ More Artificial Intelligence (AI) weather prediction (AIWP) models often produce "blurry" precipitation forecasts that overestimate drizzle and underestimate extremes. This study provides a novel solution to tackle this problem -- integrating terrain-following coordinates with global mass and energy conservation schemes into AIWP models. Forecast experiments are conducted to evaluate the effectiveness of this solution using FuXi, an example AIWP model, adapted to 1.0-degree grid spacing data. Verification results show large performance gains. The conservation schemes are found to reduce drizzle bias, whereas using terrain-following coordinates improves the estimation of extreme events and precipitation intensity spectra. Furthermore, a case study reveals that terrain-following coordinates capture near-surface winds better over mountains, offering AIWP models more accurate information on understanding the dynamics of precipitation processes. The proposed solution of this study can benefit a wide range of AIWP models and bring insights into how atmospheric domain knowledge can support the development of AIWP models. △ Less

Submitted 17 March, 2025; v1 submitted 28 February, 2025; originally announced March 2025.

Community Research Earth Digital Intelligence Twin (CREDIT)

Authors: John Schreck, Yingkai Sha, William Chapman, Dhamma Kimpara, Judith Berner, Seth McGinnis, Arnold Kazadi, Negin Sobhani, Ben Kirk, David John Gagne II

Abstract: Recent advancements in artificial intelligence (AI) for numerical weather prediction (NWP) have significantly transformed atmospheric modeling. AI NWP models outperform traditional physics-based systems, such as the Integrated Forecast System (IFS), across several global metrics while requiring fewer computational resources. However, existing AI NWP models face limitations related to training data… ▽ More Recent advancements in artificial intelligence (AI) for numerical weather prediction (NWP) have significantly transformed atmospheric modeling. AI NWP models outperform traditional physics-based systems, such as the Integrated Forecast System (IFS), across several global metrics while requiring fewer computational resources. However, existing AI NWP models face limitations related to training datasets and timestep choices, often resulting in artifacts that reduce model performance. To address these challenges, we introduce the Community Research Earth Digital Intelligence Twin (CREDIT) framework, developed at NSF NCAR. CREDIT provides a flexible, scalable, and user-friendly platform for training and deploying AI-based atmospheric models on high-performance computing systems. It offers an end-to-end pipeline for data preprocessing, model training, and evaluation, democratizing access to advanced AI NWP capabilities. We demonstrate CREDIT's potential through WXFormer, a novel deterministic vision transformer designed to predict atmospheric states autoregressively, addressing common AI NWP issues like compounding error growth with techniques such as spectral normalization, padding, and multi-step training. Additionally, to illustrate CREDIT's flexibility and state-of-the-art model comparisons, we train the FUXI architecture within this framework. Our findings show that both FUXI and WXFormer, trained on six-hourly ERA5 hybrid sigma-pressure levels, generally outperform IFS HRES in 10-day forecasts, offering potential improvements in efficiency and forecast accuracy. CREDIT's modular design enables researchers to explore various models, datasets, and training configurations, fostering innovation within the scientific community. △ Less

Submitted 8 November, 2024; originally announced November 2024.

Skin Cancer Machine Learning Model Tone Bias

Authors: James Pope, Md Hassanuzzaman, William Chapman, Huw Day, Mingmar Sherpa, Omar Emara, Nirmala Adhikari, Ayush Joshi

Abstract: Background: Many open-source skin cancer image datasets are the result of clinical trials conducted in countries with lighter skin tones. Due to this tone imbalance, machine learning models derived from these datasets can perform well at detecting skin cancer for lighter skin tones. Any tone bias in these models could introduce fairness concerns and reduce public trust in the artificial intelligen… ▽ More Background: Many open-source skin cancer image datasets are the result of clinical trials conducted in countries with lighter skin tones. Due to this tone imbalance, machine learning models derived from these datasets can perform well at detecting skin cancer for lighter skin tones. Any tone bias in these models could introduce fairness concerns and reduce public trust in the artificial intelligence health field. Methods: We examine a subset of images from the International Skin Imaging Collaboration (ISIC) archive that provide tone information. The subset has a significant tone imbalance. These imbalances could explain a model's tone bias. To address this, we train models using the imbalanced dataset and a balanced dataset to compare against. The datasets are used to train a deep convolutional neural network model to classify the images as malignant or benign. We then evaluate the models' disparate impact, based on selection rate, relative to dark or light skin tone. Results: Using the imbalanced dataset, we found that the model is significantly better at detecting malignant images in lighter tone resulting in a disparate impact of 0.577. Using the balanced dataset, we found that the model is also significantly better at detecting malignant images in lighter versus darker tones with a disparate impact of 0.684. Using the imbalanced or balanced dataset to train the model still results in a disparate impact well below the standard threshold of 0.80 which suggests the model is biased with respect to skin tone. Conclusion: The results show that typical skin cancer machine learning models can be tone biased. These results provide evidence that diagnosis or tone imbalance is not the cause of the bias. Other techniques will be necessary to identify and address the bias in these models, an area of future investigation. △ Less

Submitted 19 March, 2025; v1 submitted 8 October, 2024; originally announced October 2024.

Evidential Deep Learning: Enhancing Predictive Uncertainty Estimation for Earth System Science Applications

Authors: John S. Schreck, David John Gagne II, Charlie Becker, William E. Chapman, Kim Elmore, Da Fan, Gabrielle Gantos, Eliot Kim, Dhamma Kimpara, Thomas Martin, Maria J. Molina, Vanessa M. Pryzbylo, Jacob Radford, Belen Saavedra, Justin Willson, Christopher Wirz

Abstract: Robust quantification of predictive uncertainty is critical for understanding factors that drive weather and climate outcomes. Ensembles provide predictive uncertainty estimates and can be decomposed physically, but both physics and machine learning ensembles are computationally expensive. Parametric deep learning can estimate uncertainty with one model by predicting the parameters of a probabilit… ▽ More Robust quantification of predictive uncertainty is critical for understanding factors that drive weather and climate outcomes. Ensembles provide predictive uncertainty estimates and can be decomposed physically, but both physics and machine learning ensembles are computationally expensive. Parametric deep learning can estimate uncertainty with one model by predicting the parameters of a probability distribution but do not account for epistemic uncertainty.. Evidential deep learning, a technique that extends parametric deep learning to higher-order distributions, can account for both aleatoric and epistemic uncertainty with one model. This study compares the uncertainty derived from evidential neural networks to those obtained from ensembles. Through applications of classification of winter precipitation type and regression of surface layer fluxes, we show evidential deep learning models attaining predictive accuracy rivaling standard methods, while robustly quantifying both sources of uncertainty. We evaluate the uncertainty in terms of how well the predictions are calibrated and how well the uncertainty correlates with prediction error. Analyses of uncertainty in the context of the inputs reveal sensitivities to underlying meteorological processes, facilitating interpretation of the models. The conceptual simplicity, interpretability, and computational efficiency of evidential neural networks make them highly extensible, offering a promising approach for reliable and practical uncertainty quantification in Earth system science modeling. In order to encourage broader adoption of evidential deep learning in Earth System Science, we have developed a new Python package, MILES-GUESS (https://github.com/ai2es/miles-guess), that enables users to train and evaluate both evidential and ensemble deep learning. △ Less

Submitted 19 February, 2024; v1 submitted 22 September, 2023; originally announced September 2023.

Validation of non-negative matrix factorization for assessment of atomic pair-distribution function (PDF) data in a real-time streaming context

Authors: Chia-Hao Liu, Christopher J. Wright, Ran Gu, Sasaank Bandi, Allison Wustrow, Paul K. Todd, Daniel O'Nolan, Michelle L. Beauvais, James R. Neilson, Peter J. Chupas, Karena W. Chapman, Simon J. L. Billinge

Abstract: We validate the use of matrix factorization for the automatic identification of relevant components from atomic pair distribution function (PDF) data. We also present a newly developed software infrastructure for analyzing the PDF data arriving in streaming manner. We then apply two matrix factorization techniques, Principal Component Analysis (PCA) and Non-negative Matrix Factorization (NMF), to… ▽ More We validate the use of matrix factorization for the automatic identification of relevant components from atomic pair distribution function (PDF) data. We also present a newly developed software infrastructure for analyzing the PDF data arriving in streaming manner. We then apply two matrix factorization techniques, Principal Component Analysis (PCA) and Non-negative Matrix Factorization (NMF), to study simulated and experiment datasets in the context of in situ experiment. △ Less

Submitted 22 October, 2020; originally announced October 2020.

Direct and Indirect Effects -- An Information Theoretic Perspective

Authors: Gabriel Schamberg, William Chapman, Shang-Ping Xie, Todd P. Coleman

Abstract: Information theoretic (IT) approaches to quantifying causal influences have experienced some popularity in the literature, in both theoretical and applied (e.g. neuroscience and climate science) domains. While these causal measures are desirable in that they are model agnostic and can capture non-linear interactions, they are fundamentally different from common statistical notions of causal influe… ▽ More Information theoretic (IT) approaches to quantifying causal influences have experienced some popularity in the literature, in both theoretical and applied (e.g. neuroscience and climate science) domains. While these causal measures are desirable in that they are model agnostic and can capture non-linear interactions, they are fundamentally different from common statistical notions of causal influence in that they (1) compare distributions over the effect rather than values of the effect and (2) are defined with respect to random variables representing a cause rather than specific values of a cause. We here present IT measures of direct, indirect, and total causal effects. The proposed measures are unlike existing IT techniques in that they enable measuring causal effects that are defined with respect to specific values of a cause while still offering the flexibility and general applicability of IT techniques. We provide an identifiability result and demonstrate application of the proposed measures in estimating the causal effect of the El Niño-Southern Oscillation on temperature anomalies in the North American Pacific Northwest. △ Less

Submitted 28 July, 2020; v1 submitted 22 December, 2019; originally announced December 2019.

More or Less? Predict the Social Influence of Malicious URLs on Social Media

Authors: Chun-Ming Lai, Xiaoyun Wang, Jon W. Chapman, Yu-Cheng Lin, Yu-Chang Ho, S. Felix Wu, Patrick McDaniel, Hasan Cam

Abstract: Users of Online Social Networks (OSNs) interact with each other more than ever. In the context of a public discussion group, people receive, read, and write comments in response to articles and postings. In the absence of access control mechanisms, OSNs are a great environment for attackers to influence others, from spreading phishing URLs, to posting fake news. Moreover, OSN user behavior can be… ▽ More Users of Online Social Networks (OSNs) interact with each other more than ever. In the context of a public discussion group, people receive, read, and write comments in response to articles and postings. In the absence of access control mechanisms, OSNs are a great environment for attackers to influence others, from spreading phishing URLs, to posting fake news. Moreover, OSN user behavior can be predicted by social science concepts which include conformity and the bandwagon effect. In this paper, we show how social recommendation systems affect the occurrence of malicious URLs on Facebook. We exploit temporal features to build a prediction framework, having greater than 75% accuracy, to predict whether the following group users' behavior will increase or not. Included in this work, we demarcate classes of URLs, including those malicious URLs classified as creating critical damage, as well as those of a lesser nature which only inflict light damage such as aggressive commercial advertisements and spam content. It is our hope that the data and analyses in this paper provide a better understanding of OSN user reactions to different categories of malicious URLs, thereby providing a way to mitigate the influence of these malicious URL attacks. △ Less

Submitted 7 December, 2018; originally announced December 2018.

Comments: 10 pages, 6 figures

Multi-View Community Detection in Facebook Public Pages

Authors: Zhige Xin, Chun-Ming Lai, Jon W. Chapman, George Barnett, S. Felix Wu

Abstract: Community detection in social networks is widely studied because of its importance in uncovering how people connect and interact. However, little attention has been given to community structure in Facebook public pages. In this study, we investigate the community detection problem in Facebook newsgroup pages. In particular, to deal with the diversity of user activities, we apply multi-view cluster… ▽ More Community detection in social networks is widely studied because of its importance in uncovering how people connect and interact. However, little attention has been given to community structure in Facebook public pages. In this study, we investigate the community detection problem in Facebook newsgroup pages. In particular, to deal with the diversity of user activities, we apply multi-view clustering to integrate different views, for example, likes on posts and likes on comments. In this study, we explore the community structure in not only a given single page but across multiple pages. The results show that our method can effectively reduce isolates and improve the quality of community structure. △ Less

Submitted 6 December, 2018; v1 submitted 23 September, 2018; originally announced September 2018.

An Interactive Tool for Natural Language Processing on Clinical Text

Authors: Gaurav Trivedi, Phuong Pham, Wendy Chapman, Rebecca Hwa, Janyce Wiebe, Harry Hochheiser

Abstract: Natural Language Processing (NLP) systems often make use of machine learning techniques that are unfamiliar to end-users who are interested in analyzing clinical records. Although NLP has been widely used in extracting information from clinical text, current systems generally do not support model revision based on feedback from domain experts. We present a prototype tool that allows end users to… ▽ More Natural Language Processing (NLP) systems often make use of machine learning techniques that are unfamiliar to end-users who are interested in analyzing clinical records. Although NLP has been widely used in extracting information from clinical text, current systems generally do not support model revision based on feedback from domain experts. We present a prototype tool that allows end users to visualize and review the outputs of an NLP system that extracts binary variables from clinical text. Our tool combines multiple visualizations to help the users understand these results and make any necessary corrections, thus forming a feedback loop and helping improve the accuracy of the NLP models. We have tested our prototype in a formative think-aloud user study with clinicians and researchers involved in colonoscopy research. Results from semi-structured interviews and a System Usability Scale (SUS) analysis show that the users are able to quickly start refining NLP models, despite having very little or no experience with machine learning. Observations from these sessions suggest revisions to the interface to better support review workflow and interpretation of results. △ Less

Submitted 7 July, 2017; v1 submitted 6 July, 2017; originally announced July 2017.

Comments: 8 pages, 2 figures, 2 tables, Presented at IUI TextVis 2015 Workshop