-
Divergent Domains, Convergent Grading: Enhancing Generalization in Diabetic Retinopathy Grading
Authors:
Sharon Chokuwa,
Muhammad Haris Khan
Abstract:
Diabetic Retinopathy (DR) constitutes 5% of global blindness cases. While numerous deep learning approaches have sought to enhance traditional DR grading methods, they often falter when confronted with new out-of-distribution data thereby impeding their widespread application. In this study, we introduce a novel deep learning method for achieving domain generalization (DG) in DR grading and make t…
▽ More
Diabetic Retinopathy (DR) constitutes 5% of global blindness cases. While numerous deep learning approaches have sought to enhance traditional DR grading methods, they often falter when confronted with new out-of-distribution data thereby impeding their widespread application. In this study, we introduce a novel deep learning method for achieving domain generalization (DG) in DR grading and make the following contributions. First, we propose a new way of generating image-to-image diagnostically relevant fundus augmentations conditioned on the grade of the original fundus image. These augmentations are tailored to emulate the types of shifts in DR datasets thus increase the model's robustness. Second, we address the limitations of the standard classification loss in DG for DR fundus datasets by proposing a new DG-specific loss, domain alignment loss; which ensures that the feature vectors from all domains corresponding to the same class converge onto the same manifold for better domain generalization. Third, we tackle the coupled problem of data imbalance across DR domains and classes by proposing to employ Focal loss which seamlessly integrates with our new alignment loss. Fourth, due to inevitable observer variability in DR diagnosis that induces label noise, we propose leveraging self-supervised pretraining. This approach ensures that our DG model remains robust against early susceptibility to label noise, even when only a limited dataset of non-DR fundus images is available for pretraining. Our method demonstrates significant improvements over the strong Empirical Risk Minimization baseline and other recently proposed state-of-the-art DG methods for DR grading. Code is available at https://github.com/sharonchokuwa/dg-adr.
△ Less
Submitted 4 November, 2024;
originally announced November 2024.
-
Depth Attention for Robust RGB Tracking
Authors:
Yu Liu,
Arif Mahmood,
Muhammad Haris Khan
Abstract:
RGB video object tracking is a fundamental task in computer vision. Its effectiveness can be improved using depth information, particularly for handling motion-blurred target. However, depth information is often missing in commonly used tracking benchmarks. In this work, we propose a new framework that leverages monocular depth estimation to counter the challenges of tracking targets that are out…
▽ More
RGB video object tracking is a fundamental task in computer vision. Its effectiveness can be improved using depth information, particularly for handling motion-blurred target. However, depth information is often missing in commonly used tracking benchmarks. In this work, we propose a new framework that leverages monocular depth estimation to counter the challenges of tracking targets that are out of view or affected by motion blur in RGB video sequences. Specifically, our work introduces following contributions. To the best of our knowledge, we are the first to propose a depth attention mechanism and to formulate a simple framework that allows seamlessly integration of depth information with state of the art tracking algorithms, without RGB-D cameras, elevating accuracy and robustness. We provide extensive experiments on six challenging tracking benchmarks. Our results demonstrate that our approach provides consistent gains over several strong baselines and achieves new SOTA performance. We believe that our method will open up new possibilities for more sophisticated VOT solutions in real-world scenarios. Our code and models are publicly released: https://github.com/LiuYuML/Depth-Attention.
△ Less
Submitted 27 October, 2024;
originally announced October 2024.
-
Multi-modal Medical Image Fusion For Non-Small Cell Lung Cancer Classification
Authors:
Salma Hassan,
Hamad Al Hammadi,
Ibrahim Mohammed,
Muhammad Haris Khan
Abstract:
The early detection and nuanced subtype classification of non-small cell lung cancer (NSCLC), a predominant cause of cancer mortality worldwide, is a critical and complex issue. In this paper, we introduce an innovative integration of multi-modal data, synthesizing fused medical imaging (CT and PET scans) with clinical health records and genomic data. This unique fusion methodology leverages advan…
▽ More
The early detection and nuanced subtype classification of non-small cell lung cancer (NSCLC), a predominant cause of cancer mortality worldwide, is a critical and complex issue. In this paper, we introduce an innovative integration of multi-modal data, synthesizing fused medical imaging (CT and PET scans) with clinical health records and genomic data. This unique fusion methodology leverages advanced machine learning models, notably MedClip and BEiT, for sophisticated image feature extraction, setting a new standard in computational oncology. Our research surpasses existing approaches, as evidenced by a substantial enhancement in NSCLC detection and classification precision. The results showcase notable improvements across key performance metrics, including accuracy, precision, recall, and F1-score. Specifically, our leading multi-modal classifier model records an impressive accuracy of 94.04%. We believe that our approach has the potential to transform NSCLC diagnostics, facilitating earlier detection and more effective treatment planning and, ultimately, leading to superior patient outcomes in lung cancer care.
△ Less
Submitted 27 September, 2024;
originally announced September 2024.
-
Face-voice Association in Multilingual Environments (FAME) Challenge 2024 Evaluation Plan
Authors:
Muhammad Saad Saeed,
Shah Nawaz,
Muhammad Salman Tahir,
Rohan Kumar Das,
Muhammad Zaigham Zaheer,
Marta Moscati,
Markus Schedl,
Muhammad Haris Khan,
Karthik Nandakumar,
Muhammad Haroon Yousaf
Abstract:
The advancements of technology have led to the use of multimodal systems in various real-world applications. Among them, the audio-visual systems are one of the widely used multimodal systems. In the recent years, associating face and voice of a person has gained attention due to presence of unique correlation between them. The Face-voice Association in Multilingual Environments (FAME) Challenge 2…
▽ More
The advancements of technology have led to the use of multimodal systems in various real-world applications. Among them, the audio-visual systems are one of the widely used multimodal systems. In the recent years, associating face and voice of a person has gained attention due to presence of unique correlation between them. The Face-voice Association in Multilingual Environments (FAME) Challenge 2024 focuses on exploring face-voice association under a unique condition of multilingual scenario. This condition is inspired from the fact that half of the world's population is bilingual and most often people communicate under multilingual scenario. The challenge uses a dataset namely, Multilingual Audio-Visual (MAV-Celeb) for exploring face-voice association in multilingual environments. This report provides the details of the challenge, dataset, baselines and task details for the FAME Challenge.
△ Less
Submitted 22 July, 2024; v1 submitted 14 April, 2024;
originally announced April 2024.
-
Transformers in Medical Imaging: A Survey
Authors:
Fahad Shamshad,
Salman Khan,
Syed Waqas Zamir,
Muhammad Haris Khan,
Munawar Hayat,
Fahad Shahbaz Khan,
Huazhu Fu
Abstract:
Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as {de facto} operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growin…
▽ More
Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as {de facto} operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of the applications of Transformers in medical imaging covering various aspects, ranging from recently proposed architectural designs to unsolved issues. Specifically, we survey the use of Transformers in medical image segmentation, detection, classification, reconstruction, synthesis, registration, clinical report generation, and other tasks. In particular, for each of these applications, we develop taxonomy, identify application-specific challenges as well as provide insights to solve them, and highlight recent trends. Further, we provide a critical discussion of the field's current state as a whole, including the identification of key challenges, open problems, and outlining promising future directions. We hope this survey will ignite further interest in the community and provide researchers with an up-to-date reference regarding applications of Transformer models in medical imaging. Finally, to cope with the rapid development in this field, we intend to regularly update the relevant latest papers and their open-source implementations at \url{https://github.com/fahadshamshad/awesome-transformers-in-medical-imaging}.
△ Less
Submitted 24 January, 2022;
originally announced January 2022.
-
Unsupervised adulterated red-chili pepper content transformation for hyperspectral classification
Authors:
Muhammad Hussain Khan,
Zainab Saleem,
Muhammad Ahmad,
Ahmed Sohaib,
Hamail Ayaz
Abstract:
Preserving red-chili quality is of utmost importance in which the authorities demand the quality techniques to detect, classify and prevent it from the impurities. For example, salt, wheat flour, wheat bran, and rice bran contamination in grounded red chili, which typically a food, are a serious threat to people who are allergic to such items. This work presents the feasibility of utilizing visibl…
▽ More
Preserving red-chili quality is of utmost importance in which the authorities demand the quality techniques to detect, classify and prevent it from the impurities. For example, salt, wheat flour, wheat bran, and rice bran contamination in grounded red chili, which typically a food, are a serious threat to people who are allergic to such items. This work presents the feasibility of utilizing visible and near-infrared (VNIR) hyperspectral imaging (HSI) to detect and classify the aforementioned adulterants in red chili. However, adulterated red chili data annotation is a big challenge for classification because the acquisition of labeled data for real-time supervised learning is expensive in terms of cost and time. Therefore, this study, for the very first time proposes a novel approach to annotate the red chili samples using a clustering mechanism at 500~nm wavelength spectral response due to its dark appearance at a specified wavelength. Later the spectral samples are classified into pure or adulterated using one-class SVM. The classification performance achieves 99% in case of pure adulterants or red chili whereas 85% for adulterated samples. We further investigate that the single classification model is enough to detect any foreign substance in red chili pepper rather than cascading multiple PLS regression models.
△ Less
Submitted 9 November, 2019;
originally announced November 2019.
