-
Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM)
Authors:
Yang Zhou,
Chrystie Wan Ning Quek,
Jun Zhou,
Yan Wang,
Yang Bai,
Yuhe Ke,
Jie Yao,
Laura Gutierrez,
Zhen Ling Teo,
Darren Shu Jeng Ting,
Brian T. Soetikno,
Christopher S. Nielsen,
Tobias Elze,
Zengxiang Li,
Linh Le Dinh,
Lionel Tim-Ee Cheng,
Tran Nguyen Tuan Anh,
Chee Leong Cheng,
Tien Yin Wong,
Nan Liu,
Iain Beehuat Tan,
Tony Kiat Hon Lim,
Rick Siow Mong Goh,
Yong Liu,
Daniel Shu Wei Ting
Abstract:
Current artificial intelligence models for medical imaging are predominantly single modality and single disease. Attempts to create multimodal and multi-disease models have resulted in inconsistent clinical accuracy. Furthermore, training these models typically requires large, labour-intensive, well-labelled datasets. We developed MerMED-FM, a state-of-the-art multimodal, multi-specialty foundatio…
▽ More
Current artificial intelligence models for medical imaging are predominantly single modality and single disease. Attempts to create multimodal and multi-disease models have resulted in inconsistent clinical accuracy. Furthermore, training these models typically requires large, labour-intensive, well-labelled datasets. We developed MerMED-FM, a state-of-the-art multimodal, multi-specialty foundation model trained using self-supervised learning and a memory module. MerMED-FM was trained on 3.3 million medical images from over ten specialties and seven modalities, including computed tomography (CT), chest X-rays (CXR), ultrasound (US), pathology patches, color fundus photography (CFP), optical coherence tomography (OCT) and dermatology images. MerMED-FM was evaluated across multiple diseases and compared against existing foundational models. Strong performance was achieved across all modalities, with AUROCs of 0.988 (OCT); 0.982 (pathology); 0.951 (US); 0.943 (CT); 0.931 (skin); 0.894 (CFP); 0.858 (CXR). MerMED-FM has the potential to be a highly adaptable, versatile, cross-specialty foundation model that enables robust medical imaging interpretation across diverse medical disciplines.
△ Less
Submitted 30 June, 2025;
originally announced July 2025.
-
Infinite-dimensional next-generation reservoir computing
Authors:
Lyudmila Grigoryeva,
Hannah Lim Jing Ting,
Juan-Pablo Ortega
Abstract:
Next-generation reservoir computing (NG-RC) has attracted much attention due to its excellent performance in spatio-temporal forecasting of complex systems and its ease of implementation. This paper shows that NG-RC can be encoded as a kernel ridge regression that makes training efficient and feasible even when the space of chosen polynomial features is very large. Additionally, an extension to an…
▽ More
Next-generation reservoir computing (NG-RC) has attracted much attention due to its excellent performance in spatio-temporal forecasting of complex systems and its ease of implementation. This paper shows that NG-RC can be encoded as a kernel ridge regression that makes training efficient and feasible even when the space of chosen polynomial features is very large. Additionally, an extension to an infinite number of covariates is possible, which makes the methodology agnostic with respect to the lags into the past that are considered as explanatory factors, as well as with respect to the number of polynomial covariates, an important hyperparameter in traditional NG-RC. We show that this approach has solid theoretical backing and good behavior based on kernel universality properties previously established in the literature. Various numerical illustrations show that these generalizations of NG-RC outperform the traditional approach in several forecasting applications.
△ Less
Submitted 21 February, 2025; v1 submitted 12 December, 2024;
originally announced December 2024.
-
Sphractal: Estimating the Fractal Dimension of Surfaces Computed from Precise Atomic Coordinates via Box-Counting Algorithm
Authors:
Jonathan Yik Chang Ting,
Andrew Thomas Agars Wood,
Amanda Susan Barnard
Abstract:
The fractal dimension of a surface allows its degree of roughness to be characterized quantitatively. However, limited effort is attempted to calculate the fractal dimension of surfaces computed from precisely known atomic coordinates from computational biomolecular and nanomaterial studies. This work proposes methods to estimate the fractal dimension of the surface of any 3D object composed of sp…
▽ More
The fractal dimension of a surface allows its degree of roughness to be characterized quantitatively. However, limited effort is attempted to calculate the fractal dimension of surfaces computed from precisely known atomic coordinates from computational biomolecular and nanomaterial studies. This work proposes methods to estimate the fractal dimension of the surface of any 3D object composed of spheres, by representing the surface as either a voxelized point cloud or a mathematically exact surface, and computing its box-counting dimension. Sphractal is published as a Python package that provides these functionalities, and its utility is demonstrated on a set of simulated palladium nanoparticle data.
△ Less
Submitted 10 March, 2024; v1 submitted 22 January, 2024;
originally announced January 2024.
-
Revisiting Random Forests in a Comparative Evaluation of Graph Convolutional Neural Network Variants for Traffic Prediction
Authors:
Ta Jiun Ting,
Xiaocan Li,
Scott Sanner,
Baher Abdulhai
Abstract:
Traffic prediction is a spatiotemporal predictive task that plays an essential role in intelligent transportation systems. Today, graph convolutional neural networks (GCNNs) have become the prevailing models in the traffic prediction literature since they excel at extracting spatial correlations. In this work, we classify the components of successful GCNN prediction models and analyze the effects…
▽ More
Traffic prediction is a spatiotemporal predictive task that plays an essential role in intelligent transportation systems. Today, graph convolutional neural networks (GCNNs) have become the prevailing models in the traffic prediction literature since they excel at extracting spatial correlations. In this work, we classify the components of successful GCNN prediction models and analyze the effects of matrix factorization, attention mechanism, and weight sharing on their performance. Furthermore, we compare these variations against random forests, a traditional regression method that predates GCNNs by over 15 years. We evaluated these methods using simulated data of two regions in Toronto as well as real-world sensor data from selected California highways. We found that incorporating matrix factorization, attention, and location-specific model weights either individually or collectively into GCNNs can result in a better overall performance. Moreover, although random forest regression is a less compact model, it matches or exceeds the performance of all variations of GCNNs in our experiments. This suggests that the current graph convolutional methods may not be the best approach to traffic prediction and there is still room for improvement. Finally, our findings also suggest that for future research on GCNN for traffic prediction to be credible, researchers must include performance comparison to random forests.
△ Less
Submitted 29 May, 2023;
originally announced May 2023.
-
Deep Polarimetric HDR Reconstruction
Authors:
Juiwen Ting,
Moein Shakeri,
Hong Zhang
Abstract:
This paper proposes a novel learning based high-dynamic-range (HDR) reconstruction method using a polarization camera. We utilize a previous observation that polarization filters with different orientations can attenuate natural light differently, and we treat the multiple images acquired by the polarization camera as a set acquired under different exposure times, to introduce the development of s…
▽ More
This paper proposes a novel learning based high-dynamic-range (HDR) reconstruction method using a polarization camera. We utilize a previous observation that polarization filters with different orientations can attenuate natural light differently, and we treat the multiple images acquired by the polarization camera as a set acquired under different exposure times, to introduce the development of solutions for the HDR reconstruction problem. We propose a deep HDR reconstruction framework with a feature masking mechanism that uses polarimetric cues available from the polarization camera, called Deep Polarimetric HDR Reconstruction (DPHR). The proposed DPHR obtains polarimetric information to propagate valid features through the network more effectively to regress the missing pixels. We demonstrate through both qualitative and quantitative evaluations that the proposed DPHR performs favorably than state-of-the-art HDR reconstruction algorithms.
△ Less
Submitted 26 March, 2022;
originally announced March 2022.
-
On the Trust and Trust Modelling for the Future Fully-Connected Digital World: A Comprehensive Study
Authors:
Hannah Lim Jing Ting,
Xin Kang,
Tieyan Li,
Haiguang Wang,
Cheng-Kang Chu
Abstract:
With the fast development of digital technologies, we are running into a digital world. The relationship among people and the connections among things become more and more complex, and new challenges arise. To tackle these challenges, trust-a soft security mechanism-is considered as a promising technology. Thus, in this survey, we do a comprehensive study on the trust and trust modelling for the f…
▽ More
With the fast development of digital technologies, we are running into a digital world. The relationship among people and the connections among things become more and more complex, and new challenges arise. To tackle these challenges, trust-a soft security mechanism-is considered as a promising technology. Thus, in this survey, we do a comprehensive study on the trust and trust modelling for the future digital world. We revisit the definitions and properties of trust, and analysis the trust theories and discuss their impact on digital trust modelling. We analyze the digital world and its corresponding environment where people, things, and infrastructure connect with each other. We detail the challenges that require trust in these digital scenarios. Under our analysis of trust and the digital world, we define different types of trust relationships and find out the factors that are needed to ensure a fully representative model. Next, to meet the challenges of digital trust modelling, comprehensive trust model evaluation criteria are proposed, and potential securities and privacy issues of trust modelling are analyzed. Finally, we provide a wide-ranging analysis of different methodologies, mathematical theories, and how they can be applied to trust modelling.
△ Less
Submitted 14 June, 2021;
originally announced June 2021.
-
Deep Snapshot HDR Reconstruction Based on the Polarization Camera
Authors:
Juiwen Ting,
Xuesong Wu,
Kangkang Hu,
Hong Zhang
Abstract:
The recent development of the on-chip micro-polarizer technology has made it possible to acquire four spatially aligned and temporally synchronized polarization images with the same ease of operation as a conventional camera. In this paper, we investigate the use of this sensor technology in high-dynamic-range (HDR) imaging. Specifically, observing that natural light can be attenuated differently…
▽ More
The recent development of the on-chip micro-polarizer technology has made it possible to acquire four spatially aligned and temporally synchronized polarization images with the same ease of operation as a conventional camera. In this paper, we investigate the use of this sensor technology in high-dynamic-range (HDR) imaging. Specifically, observing that natural light can be attenuated differently by varying the orientation of the polarization filter, we treat the multiple images captured by the polarization camera as a set captured under different exposure times. In our approach, we first study the relationship among polarizer orientation, degree and angle of polarization of light to the exposure time of a pixel in the polarization image. Subsequently, we propose a deep snapshot HDR reconstruction framework to recover an HDR image using the polarization images. A polarized HDR dataset is created to train and evaluate our approach. We demonstrate that our approach performs favorably against state-of-the-art HDR reconstruction algorithms.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
-
Bio-inspired Gait Imitation of Hexapod Robot Using Event-Based Vision Sensor and Spiking Neural Network
Authors:
Justin Ting,
Yan Fang,
Ashwin Sanjay Lele,
Arijit Raychowdhury
Abstract:
Learning how to walk is a sophisticated neurological task for most animals. In order to walk, the brain must synthesize multiple cortices, neural circuits, and diverse sensory inputs. Some animals, like humans, imitate surrounding individuals to speed up their learning. When humans watch their peers, visual data is processed through a visual cortex in the brain. This complex problem of imitation-b…
▽ More
Learning how to walk is a sophisticated neurological task for most animals. In order to walk, the brain must synthesize multiple cortices, neural circuits, and diverse sensory inputs. Some animals, like humans, imitate surrounding individuals to speed up their learning. When humans watch their peers, visual data is processed through a visual cortex in the brain. This complex problem of imitation-based learning forms associations between visual data and muscle actuation through Central Pattern Generation (CPG). Reproducing this imitation phenomenon on low power, energy-constrained robots that are learning to walk remains challenging and unexplored. We propose a bio-inspired feed-forward approach based on neuromorphic computing and event-based vision to address the gait imitation problem. The proposed method trains a "student" hexapod to walk by watching an "expert" hexapod moving its legs. The student processes the flow of Dynamic Vision Sensor (DVS) data with a one-layer Spiking Neural Network (SNN). The SNN of the student successfully imitates the expert within a small convergence time of ten iterations and exhibits energy efficiency at the sub-microjoule level.
△ Less
Submitted 11 April, 2020;
originally announced April 2020.
-
Learning to Walk: Spike Based Reinforcement Learning for Hexapod Robot Central Pattern Generation
Authors:
Ashwin Sanjay Lele,
Yan Fang,
Justin Ting,
Arijit Raychowdhury
Abstract:
Learning to walk -- i.e., learning locomotion under performance and energy constraints continues to be a challenge in legged robotics. Methods such as stochastic gradient, deep reinforcement learning (RL) have been explored for bipeds, quadrupeds and hexapods. These techniques are computationally intensive and often prohibitive for edge applications. These methods rely on complex sensors and pre-p…
▽ More
Learning to walk -- i.e., learning locomotion under performance and energy constraints continues to be a challenge in legged robotics. Methods such as stochastic gradient, deep reinforcement learning (RL) have been explored for bipeds, quadrupeds and hexapods. These techniques are computationally intensive and often prohibitive for edge applications. These methods rely on complex sensors and pre-processing of data, which further increases energy and latency. Recent advances in spiking neural networks (SNNs) promise a significant reduction in computing owing to the sparse firing of neuros and has been shown to integrate reinforcement learning mechanisms with biologically observed spike time dependent plasticity (STDP). However, training a legged robot to walk by learning the synchronization patterns of central pattern generators (CPG) in an SNN framework has not been shown. This can marry the efficiency of SNNs with synchronized locomotion of CPG based systems providing breakthrough end-to-end learning in mobile robotics. In this paper, we propose a reinforcement based stochastic weight update technique for training a spiking CPG. The whole system is implemented on a lightweight raspberry pi platform with integrated sensors, thus opening up exciting new possibilities.
△ Less
Submitted 22 March, 2020;
originally announced March 2020.
-
An EMG Gesture Recognition System with Flexible High-Density Sensors and Brain-Inspired High-Dimensional Classifier
Authors:
Ali Moin,
Andy Zhou,
Abbas Rahimi,
Simone Benatti,
Alisha Menon,
Senam Tamakloe,
Jonathan Ting,
Natasha Yamamoto,
Yasser Khan,
Fred Burghardt,
Luca Benini,
Ana C. Arias,
Jan M. Rabaey
Abstract:
EMG-based gesture recognition shows promise for human-machine interaction. Systems are often afflicted by signal and electrode variability which degrades performance over time. We present an end-to-end system combating this variability using a large-area, high-density sensor array and a robust classification algorithm. EMG electrodes are fabricated on a flexible substrate and interfaced to a custo…
▽ More
EMG-based gesture recognition shows promise for human-machine interaction. Systems are often afflicted by signal and electrode variability which degrades performance over time. We present an end-to-end system combating this variability using a large-area, high-density sensor array and a robust classification algorithm. EMG electrodes are fabricated on a flexible substrate and interfaced to a custom wireless device for 64-channel signal acquisition and streaming. We use brain-inspired high-dimensional (HD) computing for processing EMG features in one-shot learning. The HD algorithm is tolerant to noise and electrode misplacement and can quickly learn from few gestures without gradient descent or back-propagation. We achieve an average classification accuracy of 96.64% for five gestures, with only 7% degradation when training and testing across different days. Our system maintains this accuracy when trained with only three trials of gestures; it also demonstrates comparable accuracy with the state-of-the-art when trained with one trial.
△ Less
Submitted 5 April, 2018; v1 submitted 27 February, 2018;
originally announced February 2018.
-
Causalities of the Taiwan Stock Market
Authors:
Juhi-Lian Julian Ting
Abstract:
Volatility, fitting with first order Landau expansion, stationarity, and causality of the Taiwan stock market (TAIEX) are investigated based on daily records. Instead of consensuses that consider stock market index change as a random time series we propose the market change as a dual time series consists of the index and the corresponding volume. Therefore, causalities between these two time ser…
▽ More
Volatility, fitting with first order Landau expansion, stationarity, and causality of the Taiwan stock market (TAIEX) are investigated based on daily records. Instead of consensuses that consider stock market index change as a random time series we propose the market change as a dual time series consists of the index and the corresponding volume. Therefore, causalities between these two time series are investigated.
△ Less
Submitted 5 April, 2003;
originally announced April 2003.
