-
MedBookVQA: A Systematic and Comprehensive Medical Benchmark Derived from Open-Access Book
Authors:
Sau Lai Yip,
Sunan He,
Yuxiang Nie,
Shu Pui Chan,
Yilin Ye,
Sum Ying Lam,
Hao Chen
Abstract:
The accelerating development of general medical artificial intelligence (GMAI), powered by multimodal large language models (MLLMs), offers transformative potential for addressing persistent healthcare challenges, including workforce deficits and escalating costs. The parallel development of systematic evaluation benchmarks emerges as a critical imperative to enable performance assessment and prov…
▽ More
The accelerating development of general medical artificial intelligence (GMAI), powered by multimodal large language models (MLLMs), offers transformative potential for addressing persistent healthcare challenges, including workforce deficits and escalating costs. The parallel development of systematic evaluation benchmarks emerges as a critical imperative to enable performance assessment and provide technological guidance. Meanwhile, as an invaluable knowledge source, the potential of medical textbooks for benchmark development remains underexploited. Here, we present MedBookVQA, a systematic and comprehensive multimodal benchmark derived from open-access medical textbooks. To curate this benchmark, we propose a standardized pipeline for automated extraction of medical figures while contextually aligning them with corresponding medical narratives. Based on this curated data, we generate 5,000 clinically relevant questions spanning modality recognition, disease classification, anatomical identification, symptom diagnosis, and surgical procedures. A multi-tier annotation system categorizes queries through hierarchical taxonomies encompassing medical imaging modalities (42 categories), body anatomies (125 structures), and clinical specialties (31 departments), enabling nuanced analysis across medical subdomains. We evaluate a wide array of MLLMs, including proprietary, open-sourced, medical, and reasoning models, revealing significant performance disparities across task types and model categories. Our findings highlight critical capability gaps in current GMAI systems while establishing textbook-derived multimodal benchmarks as essential evaluation tools. MedBookVQA establishes textbook-derived benchmarking as a critical paradigm for advancing clinical AI, exposing limitations in GMAI systems while providing anatomically structured performance metrics across specialties.
△ Less
Submitted 1 June, 2025;
originally announced June 2025.
-
Integrating Identity-Based Identification against Adaptive Adversaries in Federated Learning
Authors:
Jakub Kacper Szelag,
Ji-Jian Chin,
Lauren Ansell,
Sook-Chin Yip
Abstract:
Federated Learning (FL) has recently emerged as a promising paradigm for privacy-preserving, distributed machine learning. However, FL systems face significant security threats, particularly from adaptive adversaries capable of modifying their attack strategies to evade detection. One such threat is the presence of Reconnecting Malicious Clients (RMCs), which exploit FLs open connectivity by recon…
▽ More
Federated Learning (FL) has recently emerged as a promising paradigm for privacy-preserving, distributed machine learning. However, FL systems face significant security threats, particularly from adaptive adversaries capable of modifying their attack strategies to evade detection. One such threat is the presence of Reconnecting Malicious Clients (RMCs), which exploit FLs open connectivity by reconnecting to the system with modified attack strategies. To address this vulnerability, we propose integration of Identity-Based Identification (IBI) as a security measure within FL environments. By leveraging IBI, we enable FL systems to authenticate clients based on cryptographic identity schemes, effectively preventing previously disconnected malicious clients from re-entering the system. Our approach is implemented using the TNC-IBI (Tan-Ng-Chin) scheme over elliptic curves to ensure computational efficiency, particularly in resource-constrained environments like Internet of Things (IoT). Experimental results demonstrate that integrating IBI with secure aggregation algorithms, such as Krum and Trimmed Mean, significantly improves FL robustness by mitigating the impact of RMCs. We further discuss the broader implications of IBI in FL security, highlighting research directions for adaptive adversary detection, reputation-based mechanisms, and the applicability of identity-based cryptographic frameworks in decentralized FL architectures. Our findings advocate for a holistic approach to FL security, emphasizing the necessity of proactive defence strategies against evolving adaptive adversarial threats.
△ Less
Submitted 3 April, 2025;
originally announced April 2025.
-
Student-Powered Digital Scholarship CoLab Project in the HKUST Library: Develop a Chinese Named-Entity Recognition (NER) Tool within One Semester from the Ground Up
Authors:
Sherry S. L. Yip,
Berry L. Han,
Holly H. Y. Chan
Abstract:
Starting in February 2024, the HKUST Library further extended the scope of AI literacy to AI utilization, which focuses on fostering student involvement in utilizing state-of-the-art technologies in the projects that initiated by the Library, named "Digital Scholarship (DS) CoLab". A key focus of the DS CoLab scheme has been on cultivating talents and enabling students to utilize advanced technolo…
▽ More
Starting in February 2024, the HKUST Library further extended the scope of AI literacy to AI utilization, which focuses on fostering student involvement in utilizing state-of-the-art technologies in the projects that initiated by the Library, named "Digital Scholarship (DS) CoLab". A key focus of the DS CoLab scheme has been on cultivating talents and enabling students to utilize advanced technologies in practical context. It aims to reinforce the library's role as a catalyst and hub for fostering multidisciplinary collaboration and cultivate the "can do spirit" among university members. The Library offers 1-2 projects per year for students to engage with advanced technologies in practical contexts while supporting the Library in tackling challenges and streamlining operational tasks. The tool that introduced in this paper was mainly developed by two of the authors, Sherry Yip Sau Lai and Berry Han Liuruo, as part-time student helpers under one of our DS CoLab scheme in the 2024 Spring Semester (February to May 2024). This paper details the complete journey from ideation to implementation of developing a Chinese Named-Entity Recognition (NER) Tool from the group up within one semester, from the initial research and planning stages to execution and come up a viable product. The collaborative spirit fostered by this project, with students playing a central role, exemplifies the power and potential of innovative educational models that prioritize hands-on learning with student involvement.
△ Less
Submitted 29 March, 2025;
originally announced March 2025.
-
PSO-Net: Development of an automated psoriasis assessment system using attention-based interpretable deep neural networks
Authors:
Sharif A. Kamran,
Molly V. Lucas,
Brendon Lutnick,
Chaitanya Parmar,
Basudha Pal,
Asha Patel Shah,
David Apfel,
Steven Fakharzadeh,
Lloyd Miller,
Stephen Yip,
Kristopher Standish,
Gabriela Oana Cula
Abstract:
Psoriasis is a chronic skin condition that requires long-term treatment and monitoring. Although, the Psoriasis Area and Severity Index (PASI) is utilized as a standard measurement to assess psoriasis severity in clinical trials, it has many drawbacks such as (1) patient burden for in-person clinic visits for assessment of psoriasis, (2) time required for investigator scoring and (3) variability o…
▽ More
Psoriasis is a chronic skin condition that requires long-term treatment and monitoring. Although, the Psoriasis Area and Severity Index (PASI) is utilized as a standard measurement to assess psoriasis severity in clinical trials, it has many drawbacks such as (1) patient burden for in-person clinic visits for assessment of psoriasis, (2) time required for investigator scoring and (3) variability of inter- and intra-rater scoring. To address these drawbacks, we propose a novel and interpretable deep learning architecture called PSO-Net, which maps digital images from different anatomical regions to derive attention-based scores. Regional scores are further combined to estimate an absolute PASI score. Moreover, we devise a novel regression activation map for interpretability through ranking attention scores. Using this approach, we achieved inter-class correlation scores of 82.2% [95% CI: 77- 87%] and 87.8% [95% CI: 84-91%] with two different clinician raters, respectively.
△ Less
Submitted 30 January, 2025;
originally announced January 2025.
-
On the Security and Design of Cryptosystems Using Gabidulin-Kronecker Product Codes
Authors:
Terry Shue Chien Lau,
Zhe Sun,
Sook-Chin Yip,
Ji-Jian Chin,
Choo-Yee Ting
Abstract:
This paper is a preliminary study on the security and design of cryptosystems using Gabidulin-Kronecker Product Codes. In particular, we point out the design impracticality of the system, and propose ways to improve it.
This paper is a preliminary study on the security and design of cryptosystems using Gabidulin-Kronecker Product Codes. In particular, we point out the design impracticality of the system, and propose ways to improve it.
△ Less
Submitted 9 October, 2024;
originally announced October 2024.
-
Emergent circulation patterns from anonymized mobility data: Clustering Italy in the time of Covid
Authors:
Jules Morand,
Shoichi Yip,
Yannis Velegrakis,
Gianluca Lattanzi,
Raffaello Potestio,
Luca Tubiana
Abstract:
Using anonymized mobility data from Facebook users and publicly available information on the Italian population, we model the circulation of people in Italy before and during the early phase of the SARS-CoV-2 pandemic (COVID-19). We perform a spatial and temporal clustering of the movement network at the level of fluxes across provinces on a daily basis. The resulting partition in time successfull…
▽ More
Using anonymized mobility data from Facebook users and publicly available information on the Italian population, we model the circulation of people in Italy before and during the early phase of the SARS-CoV-2 pandemic (COVID-19). We perform a spatial and temporal clustering of the movement network at the level of fluxes across provinces on a daily basis. The resulting partition in time successfully identifies the first two lockdowns without any prior information. Similarly, the spatial clustering returns 11 to 23 clusters depending on the period ("standard" mobility vs. lockdown) using the greedy modularity communities clustering method, and 16 to 30 clusters using the critical variable selection method. Fascinatingly, the spatial clusters obtained with both methods are strongly reminiscent of the 11 regions into which emperor Augustus had divided Italy according to Pliny the Elder. This work introduces and validates a data analysis pipeline that enables us: i) to assess the reliability of data obtained from a partial and potentially biased sample of the population in performing estimates of population mobility nationwide; ii) to identify areas of a Country with well-defined mobility patterns, and iii) to distinguish different patterns from one another, resolve them in time and find their optimal spatial extent. The proposed method is generic and can be applied to other countries, with different geographical scales, and also to similar networks (e.g. biological networks). The results can thus represent a relevant step forward in the development of methods and strategies for the containment of future epidemic phenomena.
△ Less
Submitted 8 June, 2023;
originally announced June 2023.
-
A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images
Authors:
Puria Azadi Moghadam,
Sanne Van Dalen,
Karina C. Martin,
Jochen Lennerz,
Stephen Yip,
Hossein Farahani,
Ali Bashashati
Abstract:
Visual microscopic study of diseased tissue by pathologists has been the cornerstone for cancer diagnosis and prognostication for more than a century. Recently, deep learning methods have made significant advances in the analysis and classification of tissue images. However, there has been limited work on the utility of such models in generating histopathology images. These synthetic images have s…
▽ More
Visual microscopic study of diseased tissue by pathologists has been the cornerstone for cancer diagnosis and prognostication for more than a century. Recently, deep learning methods have made significant advances in the analysis and classification of tissue images. However, there has been limited work on the utility of such models in generating histopathology images. These synthetic images have several applications in pathology including utilities in education, proficiency testing, privacy, and data sharing. Recently, diffusion probabilistic models were introduced to generate high quality images. Here, for the first time, we investigate the potential use of such models along with prioritized morphology weighting and color normalization to synthesize high quality histopathology images of brain cancer. Our detailed results show that diffusion probabilistic models are capable of synthesizing a wide range of histopathology images and have superior performance compared to generative adversarial networks.
△ Less
Submitted 28 September, 2022; v1 submitted 27 September, 2022;
originally announced September 2022.
-
Automatic Estimation of Ulcerative Colitis Severity from Endoscopy Videos using Ordinal Multi-Instance Learning
Authors:
Evan Schwab,
Gabriela Oana Cula,
Kristopher Standish,
Stephen S. F. Yip,
Aleksandar Stojmirovic,
Louis Ghanem,
Christel Chehoud
Abstract:
Ulcerative colitis (UC) is a chronic inflammatory bowel disease characterized by relapsing inflammation of the large intestine. The severity of UC is often represented by the Mayo Endoscopic Subscore (MES) which quantifies mucosal disease activity from endoscopy videos. In clinical trials, an endoscopy video is assigned an MES based upon the most severe disease activity observed in the video. For…
▽ More
Ulcerative colitis (UC) is a chronic inflammatory bowel disease characterized by relapsing inflammation of the large intestine. The severity of UC is often represented by the Mayo Endoscopic Subscore (MES) which quantifies mucosal disease activity from endoscopy videos. In clinical trials, an endoscopy video is assigned an MES based upon the most severe disease activity observed in the video. For this reason, severe inflammation spread throughout the colon will receive the same MES as an otherwise healthy colon with severe inflammation restricted to a small, localized segment. Therefore, the extent of disease activity throughout the large intestine, and overall response to treatment, may not be completely captured by the MES. In this work, we aim to automatically estimate UC severity for each frame in an endoscopy video to provide a higher resolution assessment of disease activity throughout the colon. Because annotating severity at the frame-level is expensive, labor-intensive, and highly subjective, we propose a novel weakly supervised, ordinal classification method to estimate frame severity from video MES labels alone. Using clinical trial data, we first achieved 0.92 and 0.90 AUC for predicting mucosal healing and remission of UC, respectively. Then, for severity estimation, we demonstrate that our models achieve substantial Cohen's Kappa agreement with ground truth MES labels, comparable to the inter-rater agreement of expert clinicians. These findings indicate that our framework could serve as a foundation for novel clinical endpoints, based on a more localized scoring system, to better evaluate UC drug efficacy in clinical trials.
△ Less
Submitted 29 September, 2021;
originally announced September 2021.
