Showing 1–50 of 88 results for author: Belyaev, M

M3DA: Benchmark for Unsupervised Domain Adaptation in 3D Medical Image Segmentation

Authors: Boris Shirokikh, Anvar Kurmukov, Mariia Donskova, Valentin Samokhin, Mikhail Belyaev, Ivan Oseledets

Abstract: Domain shift presents a significant challenge in applying Deep Learning to the segmentation of 3D medical images from sources like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). Although numerous Domain Adaptation methods have been developed to address this issue, they are often evaluated under impractical data shift scenarios. Specifically, the medical imaging datasets used are of… ▽ More Domain shift presents a significant challenge in applying Deep Learning to the segmentation of 3D medical images from sources like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). Although numerous Domain Adaptation methods have been developed to address this issue, they are often evaluated under impractical data shift scenarios. Specifically, the medical imaging datasets used are often either private, too small for robust training and evaluation, or limited to single or synthetic tasks. To overcome these limitations, we introduce a M3DA /"mEd@/ benchmark comprising four publicly available, multiclass segmentation datasets. We have designed eight domain pairs featuring diverse and practically relevant distribution shifts. These include inter-modality shifts between MRI and CT and intra-modality shifts among various MRI acquisition parameters, different CT radiation doses, and presence/absence of contrast enhancement in images. Within the proposed benchmark, we evaluate more than ten existing domain adaptation methods. Our results show that none of them can consistently close the performance gap between the domains. For instance, the most effective method reduces the performance gap by about 62% across the tasks. This highlights the need for developing novel domain adaptation algorithms to enhance the robustness and scalability of deep learning models in medical imaging. We made our M3DA benchmark publicly available: https://github.com/BorisShirokikh/M3DA. △ Less

Submitted 24 February, 2025; originally announced February 2025.

Comments: 17 pages,7 figures,11 tables

Medical Semantic Segmentation with Diffusion Pretrain

Authors: David Li, Anvar Kurmukov, Mikhail Goncharov, Roman Sokolov, Mikhail Belyaev

Abstract: Recent advances in deep learning have shown that learning robust feature representations is critical for the success of many computer vision tasks, including medical image segmentation. In particular, both transformer and convolutional-based architectures have benefit from leveraging pretext tasks for pretraining. However, the adoption of pretext tasks in 3D medical imaging has been less explored… ▽ More Recent advances in deep learning have shown that learning robust feature representations is critical for the success of many computer vision tasks, including medical image segmentation. In particular, both transformer and convolutional-based architectures have benefit from leveraging pretext tasks for pretraining. However, the adoption of pretext tasks in 3D medical imaging has been less explored and remains a challenge, especially in the context of learning generalizable feature representations. We propose a novel pretraining strategy using diffusion models with anatomical guidance, tailored to the intricacies of 3D medical image data. We introduce an auxiliary diffusion process to pretrain a model that produce generalizable feature representations, useful for a variety of downstream segmentation tasks. We employ an additional model that predicts 3D universal body-part coordinates, providing guidance during the diffusion process and improving spatial awareness in generated representations. This approach not only aids in resolving localization inaccuracies but also enriches the model's ability to understand complex anatomical structures. Empirical validation on a 13-class organ segmentation task demonstrate the effectiveness of our pretraining technique. It surpasses existing restorative pretraining methods in 3D medical image segmentation by $7.5\%$, and is competitive with the state-of-the-art contrastive pretraining approach, achieving an average Dice coefficient of 67.8 in a non-linear evaluation scenario. △ Less

Submitted 31 January, 2025; originally announced January 2025.

Anatomical Positional Embeddings

Authors: Mikhail Goncharov, Valentin Samokhin, Eugenia Soboleva, Roman Sokolov, Boris Shirokikh, Mikhail Belyaev, Anvar Kurmukov, Ivan Oseledets

Abstract: We propose a self-supervised model producing 3D anatomical positional embeddings (APE) of individual medical image voxels. APE encodes voxels' anatomical closeness, i.e., voxels of the same organ or nearby organs always have closer positional embeddings than the voxels of more distant body parts. In contrast to the existing models of anatomical positional embeddings, our method is able to efficien… ▽ More We propose a self-supervised model producing 3D anatomical positional embeddings (APE) of individual medical image voxels. APE encodes voxels' anatomical closeness, i.e., voxels of the same organ or nearby organs always have closer positional embeddings than the voxels of more distant body parts. In contrast to the existing models of anatomical positional embeddings, our method is able to efficiently produce a map of voxel-wise embeddings for a whole volumetric input image, which makes it an optimal choice for different downstream applications. We train our APE model on 8400 publicly available CT images of abdomen and chest regions. We demonstrate its superior performance compared with the existing models on anatomical landmark retrieval and weakly-supervised few-shot localization of 13 abdominal organs. As a practical application, we show how to cheaply train APE to crop raw CT images to different anatomical regions of interest with 0.99 recall, while reducing the image volume by 10-100 times. The code and the pre-trained APE model are available at https://github.com/mishgon/ape . △ Less

Submitted 16 September, 2024; originally announced September 2024.

Objective Features Extracted from Motor Activity Time Series for Food Addiction Analysis Using Machine Learning

Authors: Mikhail Borisenkov, Andrei Velichko, Maksim Belyaev, Dmitry Korzun, Tatyana Tserne, Larisa Bakutova, Denis Gubin

Abstract: This study investigates machine learning algorithms to identify objective features for diagnosing food addiction (FA) and assessing confirmed symptoms (SC). Data were collected from 81 participants (mean age: 21.5 years, range: 18-61 years, women: 77.8%) whose FA and SC were measured using the Yale Food Addiction Scale (YFAS). Participants provided demographic and anthropometric data, completed th… ▽ More This study investigates machine learning algorithms to identify objective features for diagnosing food addiction (FA) and assessing confirmed symptoms (SC). Data were collected from 81 participants (mean age: 21.5 years, range: 18-61 years, women: 77.8%) whose FA and SC were measured using the Yale Food Addiction Scale (YFAS). Participants provided demographic and anthropometric data, completed the YFAS, the Zung Self-Rating Depression Scale, and the Dutch Eating Behavior Questionnaire, and wore an actimeter on the non-dominant wrist for a week to record motor activity. Analysis of the actimetric data identified significant statistical and entropy-based features that accurately predicted FA and SC using ML. The Matthews correlation coefficient (MCC) was the primary metric. Activity-related features were more effective for FA prediction (MCC=0.88) than rest-related features (MCC=0.68). For SC, activity segments yielded MCC=0.47, rest segments MCC=0.38, and their combination MCC=0.51. Significant correlations were also found between actimetric features related to FA, emotional, and restrained eating behaviors, supporting the model's validity. Our results support the concept of a human bionic suite composed of IoT devices and ML sensors, which implements health digital assistance with real-time monitoring and analysis of physiological indicators related to FA and SC. △ Less

Submitted 5 December, 2024; v1 submitted 30 August, 2024; originally announced September 2024.

Comments: 16 pages, 3 figures, 14 tables

Robust Curve Detection in Volumetric Medical Imaging via Attraction Field

Authors: Farukh Yaushev, Daria Nogina, Valentin Samokhin, Mariya Dugova, Ekaterina Petrash, Dmitry Sevryukov, Mikhail Belyaev, Maxim Pisov

Abstract: Understanding body part geometry is crucial for precise medical diagnostics. Curves effectively describe anatomical structures and are widely used in medical imaging applications related to cardiovascular, respiratory, and skeletal diseases. Traditional curve detection methods are often task-specific, relying heavily on domain-specific features, limiting their broader applicability. This paper int… ▽ More Understanding body part geometry is crucial for precise medical diagnostics. Curves effectively describe anatomical structures and are widely used in medical imaging applications related to cardiovascular, respiratory, and skeletal diseases. Traditional curve detection methods are often task-specific, relying heavily on domain-specific features, limiting their broader applicability. This paper introduces a novel approach for detecting non-branching curves, which does not require prior knowledge of the object's orientation, shape, or position. Our method uses neural networks to predict (1) an attraction field, which offers subpixel accuracy, and (2) a closeness map, which limits the region of interest and essentially eliminates outliers far from the desired curve. We tested our curve detector on several clinically relevant tasks with diverse morphologies and achieved impressive subpixel-level accuracy results that surpass existing methods, highlighting its versatility and robustness. Additionally, to support further advancements in this field, we provide our private annotations of aortic centerlines and masks, which can serve as a benchmark for future research. The dataset can be found at https://github.com/neuro-ml/curve-detection. △ Less

Submitted 14 August, 2024; v1 submitted 2 August, 2024; originally announced August 2024.

Comments: Accepted to ShapeMI MICCAI 2024

The impact of deep learning aid on the workload and interpretation accuracy of radiologists on chest computed tomography: a cross-over reader study

Authors: Anvar Kurmukov, Valeria Chernina, Regina Gareeva, Maria Dugova, Ekaterina Petrash, Olga Aleshina, Maxim Pisov, Boris Shirokikh, Valentin Samokhin, Vladislav Proskurov, Stanislav Shimovolos, Maria Basova, Mikhail Goncahrov, Eugenia Soboleva, Maria Donskova, Farukh Yaushev, Alexey Shevtsov, Alexey Zakharov, Talgat Saparov, Victor Gombolevskiy, Mikhail Belyaev

Abstract: Interpretation of chest computed tomography (CT) is time-consuming. Previous studies have measured the time-saving effect of using a deep-learning-based aid (DLA) for CT interpretation. We evaluated the joint impact of a multi-pathology DLA on the time and accuracy of radiologists' reading. 40 radiologists were randomly split into three experimental arms: control (10), who interpret studies with… ▽ More Interpretation of chest computed tomography (CT) is time-consuming. Previous studies have measured the time-saving effect of using a deep-learning-based aid (DLA) for CT interpretation. We evaluated the joint impact of a multi-pathology DLA on the time and accuracy of radiologists' reading. 40 radiologists were randomly split into three experimental arms: control (10), who interpret studies without assistance; informed group (10), who were briefed about DLA pathologies, but performed readings without it; and the experimental group (20), who interpreted half studies with DLA, and half without. Every arm used the same 200 CT studies retrospectively collected from BIMCV-COVID19 dataset; each radiologist provided readings for 20 CT studies. We compared interpretation time, and accuracy of participants diagnostic report with respect to 12 pathological findings. Mean reading time per study was 15.6 minutes [SD 8.5] in the control arm, 13.2 minutes [SD 8.7] in the informed arm, 14.4 [SD 10.3] in the experimental arm without DLA, and 11.4 minutes [SD 7.8] in the experimental arm with DLA. Mean sensitivity and specificity were 41.5 [SD 30.4], 86.8 [SD 28.3] in the control arm; 53.5 [SD 22.7], 92.3 [SD 9.4] in the informed non-assisted arm; 63.2 [SD 16.4], 92.3 [SD 8.2] in the experimental arm without DLA; and 91.6 [SD 7.2], 89.9 [SD 6.0] in the experimental arm with DLA. DLA speed up interpretation time per study by 2.9 minutes (CI95 [1.7, 4.3], p<0.0005), increased sensitivity by 28.4 (CI95 [23.4, 33.4], p<0.0005), and decreased specificity by 2.4 (CI95 [0.6, 4.3], p=0.13). Of 20 radiologists in the experimental arm, 16 have improved reading time and sensitivity, two improved their time with a marginal drop in sensitivity, and two participants improved sensitivity with increased time. Overall, DLA introduction decreased reading time by 20.6%. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 17 pages, 6 figures, 8 tables

Hierarchical Loss And Geometric Mask Refinement For Multilabel Ribs Segmentation

Authors: Aleksei Leonov, Aleksei Zakharov, Sergey Koshelev, Maxim Pisov, Anvar Kurmukov, Mikhail Belyaev

Abstract: Automatic ribs segmentation and numeration can increase computed tomography assessment speed and reduce radiologists mistakes. We introduce a model for multilabel ribs segmentation with hierarchical loss function, which enable to improve multilabel segmentation quality. Also we propose postprocessing technique to further increase labeling quality. Our model achieved new state-of-the-art 98.2% labe… ▽ More Automatic ribs segmentation and numeration can increase computed tomography assessment speed and reduce radiologists mistakes. We introduce a model for multilabel ribs segmentation with hierarchical loss function, which enable to improve multilabel segmentation quality. Also we propose postprocessing technique to further increase labeling quality. Our model achieved new state-of-the-art 98.2% label accuracy on public RibSeg v2 dataset, surpassing previous result by 6.7%. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Comments: Accepted to IEEE ISBI 2024

Entropy-based machine learning model for diagnosis and monitoring of Parkinson's Disease in smart IoT environment

Authors: Maksim Belyaev, Murugappan Murugappan, Andrei Velichko, Dmitry Korzun

Abstract: The study presents the concept of a computationally efficient machine learning (ML) model for diagnosing and monitoring Parkinson's disease (PD) in an Internet of Things (IoT) environment using rest-state EEG signals (rs-EEG). We computed different types of entropy from EEG signals and found that Fuzzy Entropy performed the best in diagnosing and monitoring PD using rs-EEG. We also investigated di… ▽ More The study presents the concept of a computationally efficient machine learning (ML) model for diagnosing and monitoring Parkinson's disease (PD) in an Internet of Things (IoT) environment using rest-state EEG signals (rs-EEG). We computed different types of entropy from EEG signals and found that Fuzzy Entropy performed the best in diagnosing and monitoring PD using rs-EEG. We also investigated different combinations of signal frequency ranges and EEG channels to accurately diagnose PD. Finally, with a fewer number of features (11 features), we achieved a maximum classification accuracy (ARKF) of ~99.9%. The most prominent frequency range of EEG signals has been identified, and we have found that high classification accuracy depends on low-frequency signal components (0-4 Hz). Moreover, the most informative signals were mainly received from the right hemisphere of the head (F8, P8, T8, FC6). Furthermore, we assessed the accuracy of the diagnosis of PD using three different lengths of EEG data (150-1000 samples). Because the computational complexity is reduced by reducing the input data. As a result, we have achieved a maximum mean accuracy of 99.9% for a sample length (LEEG) of 1000 (~7.8 seconds), 98.2% with a LEEG of 800 (~6.2 seconds), and 79.3% for LEEG = 150 (~1.2 seconds). By reducing the number of features and segment lengths, the computational cost of classification can be reduced. Lower-performance smart ML sensors can be used in IoT environments for enhances human resilience to PD. △ Less

Submitted 28 August, 2023; originally announced September 2023.

Comments: 19 pages, 10 figures, 2 tables

Redesigning Out-of-Distribution Detection on 3D Medical Images

Authors: Anton Vasiliuk, Daria Frolova, Mikhail Belyaev, Boris Shirokikh

Abstract: Detecting out-of-distribution (OOD) samples for trusted medical image segmentation remains a significant challenge. The critical issue here is the lack of a strict definition of abnormal data, which often results in artificial problem settings without measurable clinical impact. In this paper, we redesign the OOD detection problem according to the specifics of volumetric medical imaging and relate… ▽ More Detecting out-of-distribution (OOD) samples for trusted medical image segmentation remains a significant challenge. The critical issue here is the lack of a strict definition of abnormal data, which often results in artificial problem settings without measurable clinical impact. In this paper, we redesign the OOD detection problem according to the specifics of volumetric medical imaging and related downstream tasks (e.g., segmentation). We propose using the downstream model's performance as a pseudometric between images to define abnormal samples. This approach enables us to weigh different samples based on their performance impact without an explicit ID/OOD distinction. We incorporate this weighting in a new metric called Expected Performance Drop (EPD). EPD is our core contribution to the new problem design, allowing us to rank methods based on their clinical impact. We demonstrate the effectiveness of EPD-based evaluation in 11 CT and MRI OOD detection challenges. △ Less

Submitted 7 August, 2023; originally announced August 2023.

vox2vec: A Framework for Self-supervised Contrastive Learning of Voxel-level Representations in Medical Images

Authors: Mikhail Goncharov, Vera Soboleva, Anvar Kurmukov, Maxim Pisov, Mikhail Belyaev

Abstract: This paper introduces vox2vec - a contrastive method for self-supervised learning (SSL) of voxel-level representations. vox2vec representations are modeled by a Feature Pyramid Network (FPN): a voxel representation is a concatenation of the corresponding feature vectors from different pyramid levels. The FPN is pre-trained to produce similar representations for the same voxel in different augmente… ▽ More This paper introduces vox2vec - a contrastive method for self-supervised learning (SSL) of voxel-level representations. vox2vec representations are modeled by a Feature Pyramid Network (FPN): a voxel representation is a concatenation of the corresponding feature vectors from different pyramid levels. The FPN is pre-trained to produce similar representations for the same voxel in different augmented contexts and distinctive representations for different voxels. This results in unified multi-scale representations that capture both global semantics (e.g., body part) and local semantics (e.g., different small organs or healthy versus tumor tissue). We use vox2vec to pre-train a FPN on more than 6500 publicly available computed tomography images. We evaluate the pre-trained representations by attaching simple heads on top of them and training the resulting models for 22 segmentation tasks. We show that vox2vec outperforms existing medical imaging SSL techniques in three evaluation setups: linear and non-linear probing and end-to-end fine-tuning. Moreover, a non-linear head trained on top of the frozen vox2vec representations achieves competitive performance with the FPN trained from scratch while having 50 times fewer trainable parameters. The code is available at https://github.com/mishgon/vox2vec . △ Less

Submitted 27 July, 2023; originally announced July 2023.

Comments: MICCAI 2023

Limitations of Out-of-Distribution Detection in 3D Medical Image Segmentation

Authors: Anton Vasiliuk, Daria Frolova, Mikhail Belyaev, Boris Shirokikh

Abstract: Deep Learning models perform unreliably when the data comes from a distribution different from the training one. In critical applications such as medical imaging, out-of-distribution (OOD) detection methods help to identify such data samples, preventing erroneous predictions. In this paper, we further investigate the OOD detection effectiveness when applied to 3D medical image segmentation. We des… ▽ More Deep Learning models perform unreliably when the data comes from a distribution different from the training one. In critical applications such as medical imaging, out-of-distribution (OOD) detection methods help to identify such data samples, preventing erroneous predictions. In this paper, we further investigate the OOD detection effectiveness when applied to 3D medical image segmentation. We design several OOD challenges representing clinically occurring cases and show that none of these methods achieve acceptable performance. Methods not dedicated to segmentation severely fail to perform in the designed setups; their best mean false positive rate at 95% true positive rate (FPR) is 0.59. Segmentation-dedicated ones still achieve suboptimal performance, with the best mean FPR of 0.31 (lower is better). To indicate this suboptimality, we develop a simple method called Intensity Histogram Features (IHF), which performs comparable or better in the same challenges, with a mean FPR of 0.25. Our findings highlight the limitations of the existing OOD detection methods on 3D medical images and present a promising avenue for improving them. To facilitate research in this area, we release the designed challenges as a publicly available benchmark and formulate practical criteria to test the OOD detection generalization beyond the suggested benchmark. We also propose IHF as a solid baseline to contest the emerging methods. △ Less

Submitted 23 June, 2023; originally announced June 2023.

Comments: This work has been submitted to the IEEE for possible publication. 10 pages, 5 figures, 5 tables

Solar Active Regions Detection Via 2D Circular Kernel Time Series Transformation, Entropy and Machine Learning Approach

Authors: Irewola Aaron Oludehinwa, Andrei Velichko, Maksim Belyaev, Olasunkanmi I. Olusola

Abstract: This study proposes an enhancement to the existing method for detecting Solar Active Regions (ARs). Our technique tracks ARs using images from the Atmospheric Imaging Assembly (AIA) of NASA's Solar Dynamics Observatory (SDO). It involves a 2D circular kernel time series transformation, combined with Statistical and Entropy measures, and a Machine Learning (ML) approach. The technique transforms th… ▽ More This study proposes an enhancement to the existing method for detecting Solar Active Regions (ARs). Our technique tracks ARs using images from the Atmospheric Imaging Assembly (AIA) of NASA's Solar Dynamics Observatory (SDO). It involves a 2D circular kernel time series transformation, combined with Statistical and Entropy measures, and a Machine Learning (ML) approach. The technique transforms the circular area around pixels in the SDO AIA images into one-dimensional time series (1-DTS). Statistical measures (Median Value, Xmed; 95th Percentile, X95) and Entropy measures (Distribution Entropy, DisEn; Fuzzy Entropy, FuzzyEn) are used as feature selection methods (FSM 1), alongside a method applying 1-DTS elements directly as features (FSM 2). The ML algorithm classifies these series into three categories: no Active Region (nARs type 1, class 1), non-flaring Regions outside active regions with brightness (nARs type 2, class 2), and flaring Active Regions (ARs, class 3). The ML model achieves a classification accuracy of 0.900 and 0.914 for Entropy and Statistical measures, respectively. Notably, Fuzzy Entropy shows the highest classification accuracy (AKF=0.895), surpassing DisEn (AKF=0.738), X95 (AKF=0.873), and Xmed (AKF=0.840). This indicates the high effectiveness of Entropy and Statistical measures for AR detection in SDO AIA images. FSM 2 captures a similar distribution of flaring AR activities as FSM 1. Additionally, we introduce a generalizing characteristic of AR activities (GSA), finding a direct agreement between increased AR activities and higher GSA values. The Python code implementation of the proposed method is available in supplementary material. △ Less

Submitted 26 August, 2024; v1 submitted 14 June, 2023; originally announced June 2023.

Comments: 30 pages, 10 figures, 4 tables

A Bio-Inspired Chaos Sensor Model Based on the Perceptron Neural Network: Machine Learning Concept and Application for Computational Neuro-Science

Authors: Andrei Velichko, Petr Boriskov, Maksim Belyaev, Vadim Putrolaynen

Abstract: The study presents a bio-inspired chaos sensor model based on the perceptron neural network for the estimation of entropy of spike train in neurodynamic systems. After training, the sensor on perceptron, having 50 neurons in the hidden layer and 1 neuron at the output, approximates the fuzzy entropy of a short time series with high accuracy, with a determination coefficient of R2 ~ 0.9. The Hindma… ▽ More The study presents a bio-inspired chaos sensor model based on the perceptron neural network for the estimation of entropy of spike train in neurodynamic systems. After training, the sensor on perceptron, having 50 neurons in the hidden layer and 1 neuron at the output, approximates the fuzzy entropy of a short time series with high accuracy, with a determination coefficient of R2 ~ 0.9. The Hindmarsh-Rose spike model was used to generate time series of spike intervals, and datasets for training and testing the perceptron. The selection of the hyperparameters of the perceptron model and the estimation of the sensor accuracy were performed using the K-block cross-validation method. Even for a hidden layer with one neuron, the model approximates the fuzzy entropy with good results and the metric R2 ~ 0.5-0.8. In a simplified model with one neuron and equal weights in the first layer, the principle of approximation is based on the linear transformation of the average value of the time series into the entropy value. An example of using the chaos sensor on spike train of action potential recordings from the L5 dorsal rootlet of rat is provided. The bio-inspired chaos sensor model based on an ensemble of neurons is able to dynamically track the chaotic behavior of a spike signal and transmit this information to other parts of the neurodynamic model for further processing. The study will be useful for specialists in the field of computational neuroscience, and also to create humanoid and animal robots, and bio-robots with limited resources. △ Less

Submitted 15 August, 2023; v1 submitted 2 June, 2023; originally announced June 2023.

Comments: 28 pages, 15 figures, 5 tables

Journal ref: Sensors 2023, 23, 7137

Neural Network Entropy (NNetEn): Entropy-Based EEG Signal and Chaotic Time Series Classification, Python Package for NNetEn Calculation

Authors: Andrei Velichko, Maksim Belyaev, Yuriy Izotov, Murugappan Murugappan, Hanif Heidari

Abstract: Entropy measures are effective features for time series classification problems. Traditional entropy measures, such as Shannon entropy, use probability distribution function. However, for the effective separation of time series, new entropy estimation methods are required to characterize the chaotic dynamic of the system. Our concept of Neural Network Entropy (NNetEn) is based on the classificatio… ▽ More Entropy measures are effective features for time series classification problems. Traditional entropy measures, such as Shannon entropy, use probability distribution function. However, for the effective separation of time series, new entropy estimation methods are required to characterize the chaotic dynamic of the system. Our concept of Neural Network Entropy (NNetEn) is based on the classification of special datasets in relation to the entropy of the time series recorded in the reservoir of the neural network. NNetEn estimates the chaotic dynamics of time series in an original way and does not take into account probability distribution functions. We propose two new classification metrics: R2 Efficiency and Pearson Efficiency. The efficiency of NNetEn is verified on separation of two chaotic time series of sine mapping using dispersion analysis. For two close dynamic time series (r = 1.1918 and r = 1.2243), the F-ratio has reached the value of 124 and reflects high efficiency of the introduced method in classification problems. The electroenceph-alography signal classification for healthy persons and patients with Alzheimer disease illustrates the practical application of the NNetEn features. Our computations demonstrate the synergistic effect of increasing classification accuracy when applying traditional entropy measures and the NNetEn concept conjointly. An implementation of the algorithms in Python is presented. △ Less

Submitted 18 May, 2023; v1 submitted 31 March, 2023; originally announced March 2023.

Comments: 26 pages, 18 figures, 2 tables

Journal ref: Algorithms 2023, 16, 255

HJM Local Volatility Model

Authors: V. M. Belyaev

Abstract: Local Volatility (LV) is a powerful tool for market modeling, enabling the generation of arbitrage-free scenarios calibrated to all European options. To implement LV, we need to interpolate and extrapolate option prices. This approach is significantly faster and more accurate than any parameterized model. The implementation is demonstrated specifically for interest rate swaptions and caplets. A… ▽ More Local Volatility (LV) is a powerful tool for market modeling, enabling the generation of arbitrage-free scenarios calibrated to all European options. To implement LV, we need to interpolate and extrapolate option prices. This approach is significantly faster and more accurate than any parameterized model. The implementation is demonstrated specifically for interest rate swaptions and caplets. A key component of this method is the Small Volatility Approximation within the HJM interest rate model, which is used to calculate sensitivity of forward bond volatility. These calculations are deterministic and fast, with excellent calibration accuracy. A detailed description of the calibration procedure is provided. △ Less

Submitted 30 January, 2025; v1 submitted 31 January, 2023; originally announced January 2023.

Comments: 16 pages, 15 Figures. Added Caplets

Solving Sample-Level Out-of-Distribution Detection on 3D Medical Images

Authors: Daria Frolova, Anton Vasiliuk, Mikhail Belyaev, Boris Shirokikh

Abstract: Deep Learning (DL) models tend to perform poorly when the data comes from a distribution different from the training one. In critical applications such as medical imaging, out-of-distribution (OOD) detection helps to identify such data samples, increasing the model's reliability. Recent works have developed DL-based OOD detection that achieves promising results on 2D medical images. However, scali… ▽ More Deep Learning (DL) models tend to perform poorly when the data comes from a distribution different from the training one. In critical applications such as medical imaging, out-of-distribution (OOD) detection helps to identify such data samples, increasing the model's reliability. Recent works have developed DL-based OOD detection that achieves promising results on 2D medical images. However, scaling most of these approaches on 3D images is computationally intractable. Furthermore, the current 3D solutions struggle to achieve acceptable results in detecting even synthetic OOD samples. Such limited performance might indicate that DL often inefficiently embeds large volumetric images. We argue that using the intensity histogram of the original CT or MRI scan as embedding is descriptive enough to run OOD detection. Therefore, we propose a histogram-based method that requires no DL and achieves almost perfect results in this domain. Our proposal is supported two-fold. We evaluate the performance on the publicly available datasets, where our method scores 1.0 AUROC in most setups. And we score second in the Medical Out-of-Distribution challenge without fine-tuning and exploiting task-specific knowledge. Carefully discussing the limitations, we conclude that our method solves the sample-level OOD detection on 3D medical images in the current setting. △ Less

Submitted 23 June, 2023; v1 submitted 13 December, 2022; originally announced December 2022.

Comments: We had made a mistake in the proposed algorithm's code (IHF), which led to a biased evaluation -- the reported AUROC scores (Tab. 1) are higher than they should be. It led to a false conclusion and the primary paper's message

Exploring Structure-Wise Uncertainty for 3D Medical Image Segmentation

Authors: Anton Vasiliuk, Daria Frolova, Mikhail Belyaev, Boris Shirokikh

Abstract: When applying a Deep Learning model to medical images, it is crucial to estimate the model uncertainty. Voxel-wise uncertainty is a useful visual marker for human experts and could be used to improve the model's voxel-wise output, such as segmentation. Moreover, uncertainty provides a solid foundation for out-of-distribution (OOD) detection, improving the model performance on the image-wise level.… ▽ More When applying a Deep Learning model to medical images, it is crucial to estimate the model uncertainty. Voxel-wise uncertainty is a useful visual marker for human experts and could be used to improve the model's voxel-wise output, such as segmentation. Moreover, uncertainty provides a solid foundation for out-of-distribution (OOD) detection, improving the model performance on the image-wise level. However, one of the frequent tasks in medical imaging is the segmentation of distinct, local structures such as tumors or lesions. Here, the structure-wise uncertainty allows more precise operations than image-wise and more semantic-aware than voxel-wise. The way to produce uncertainty for individual structures remains poorly explored. We propose a framework to measure the structure-wise uncertainty and evaluate the impact of OOD data on the model performance. Thus, we identify the best UE method to improve the segmentation quality. The proposed framework is tested on three datasets with the tumor segmentation task: LIDC-IDRI, LiTS, and a private one with multiple brain metastases cases. △ Less

Submitted 1 November, 2022; originally announced November 2022.

Detection of Risk Predictors of COVID-19 Mortality with Classifier Machine Learning Models Operated with Routine Laboratory Biomarkers

Authors: Mehmet Tahir Huyut, Andrei Velichko, Maksim Belyaev

Abstract: Early evaluation of patients who require special care and who have high death-expectancy in COVID-19, and the effective determination of relevant biomarkers on large sample-groups are important to reduce mortality. This study aimed to reveal the routine blood-value predictors of COVID-19 mortality and to determine the lethal-risk levels of these predictors during the disease process. The dataset o… ▽ More Early evaluation of patients who require special care and who have high death-expectancy in COVID-19, and the effective determination of relevant biomarkers on large sample-groups are important to reduce mortality. This study aimed to reveal the routine blood-value predictors of COVID-19 mortality and to determine the lethal-risk levels of these predictors during the disease process. The dataset of the study consists of 38 routine blood-values of 2597 patients who died (n = 233) and those who recovered (n = 2364) from COVID-19 in August-December, 2021. In this study, the histogram-based gradient-boosting (HGB) model was the most successful machine-learning classifier in detecting living and deceased COVID-19 patients (with squared F1 metrics F1^2 = 1). The most efficient binary combinations with procalcitonin were obtained with D-dimer, ESR, D-Bil and ferritin. The HGB model operated with these feature pairs correctly detected almost all of the patients who survived and those who died (precision > 0.98, recall > 0.98, F1^2 > 0.98). Furthermore, in the HGB model operated with a single feature, the most efficient features were procalcitonin (F1^2 = 0.96) and ferritin (F1^2 = 0.91). In addition, according to the two-threshold approach, ferritin values between 376.2 mkg/L and 396.0 mkg/L (F1^2 = 0.91) and pro-calcitonin values between 0.2 mkg/L and 5.2 mkg/L (F1^2 = 0.95) were found to be fatal risk levels for COVID-19. Considering all the results, we suggest that many features combined with these features, especially procalcitonin and ferritin, operated with the HGB model, can be used to achieve very successful results in the classification of those who live, and those who die from COVID-19. Moreover, we strongly recommend that clinicians consider the critical levels we have found for procalcitonin and ferritin properties, to reduce the lethality of the COVID-19 disease. △ Less

Submitted 29 November, 2022; v1 submitted 22 October, 2022; originally announced October 2022.

Comments: 29 pages, 14 figures, 6 tables

Journal ref: Appl. Sci. 2022, 12, 12180

Entropy Approximation by Machine Learning Regression: Application for Irregularity Evaluation of Images in Remote Sensing

Authors: Andrei Velichko, Maksim Belyaev, Matthias P. Wagner, Alireza Taravat

Abstract: Approximation of entropies of various types using machine learning (ML) regression methods are shown for the first time. The ML models presented in this study define the complexity of the short time series by approximating dissimilar entropy techniques such as Singular value decomposition entropy (SvdEn), Permutation entropy (PermEn), Sample entropy (SampEn) and Neural Network entropy (NNetEn) and… ▽ More Approximation of entropies of various types using machine learning (ML) regression methods are shown for the first time. The ML models presented in this study define the complexity of the short time series by approximating dissimilar entropy techniques such as Singular value decomposition entropy (SvdEn), Permutation entropy (PermEn), Sample entropy (SampEn) and Neural Network entropy (NNetEn) and their 2D analogies. A new method for calculating SvdEn2D, PermEn2D and SampEn2D for 2D images was tested using the technique of circular kernels. Training and testing datasets on the basis of Sentinel-2 images are presented (two training images and one hundred and ninety-eight testing images). The results of entropy approximation are demonstrated using the example of calculating the 2D entropy of Sentinel-2 images and R^2 metric evaluation. The applicability of the method for the short time series with a length from N = 5 to N = 113 elements is shown. A tendency for the R^2 metric to decrease with an increase in the length of the time series was found. For SvdEn entropy, the regression accuracy is R^2 > 0.99 for N = 5 and R^2 > 0.82 for N = 113. The best metrics were observed for the ML_SvdEn2D and ML_NNetEn2D models. The results of the study can be used for fundamental research of entropy approximations of various types using ML regression, as well as for accelerating entropy calculations in remote sensing. The versatility of the model is shown on a synthetic chaotic time series using Planck map and logistic map. △ Less

Submitted 29 November, 2022; v1 submitted 13 October, 2022; originally announced October 2022.

Comments: 25 pages, 24 figures, 4 tables

Journal ref: Remote Sens. 2022, 14, 5983

Machine Learning Sensors for Diagnosis of COVID-19 Disease Using Routine Blood Values for Internet of Things Application

Authors: Andrei Velichko, Mehmet Tahir Huyut, Maksim Belyaev, Yuriy Izotov, Dmitry Korzun

Abstract: Healthcare digitalization requires effective applications of human sensors, when various parameters of the human body are instantly monitored in everyday life due to the Internet of Things (IoT). In particular, machine learning (ML) sensors for the prompt diagnosis of COVID-19 are an important option for IoT application in healthcare and ambient assisted living (AAL). Determining a COVID-19 infect… ▽ More Healthcare digitalization requires effective applications of human sensors, when various parameters of the human body are instantly monitored in everyday life due to the Internet of Things (IoT). In particular, machine learning (ML) sensors for the prompt diagnosis of COVID-19 are an important option for IoT application in healthcare and ambient assisted living (AAL). Determining a COVID-19 infected status with various diagnostic tests and imaging results is costly and time-consuming. This study provides a fast, reliable and cost-effective alternative tool for the diagnosis of COVID-19 based on the routine blood values (RBVs) measured at admission. The dataset of the study consists of a total of 5296 patients with the same number of negative and positive COVID-19 test results and 51 routine blood values. In this study, 13 popular classifier machine learning models and the LogNNet neural network model were exanimated. The most successful classifier model in terms of time and accuracy in the detection of the disease was the histogram-based gradient boosting (HGB) (accuracy: 100%, time: 6.39 sec). The HGB classifier identified the 11 most important features (LDL, cholesterol, HDL-C, MCHC, triglyceride, amylase, UA, LDH, CK-MB, ALP and MCH) to detect the disease with 100% accuracy. In addition, the importance of single, double and triple combinations of these features in the diagnosis of the disease was discussed. We propose to use these 11 features and their binary combinations as important biomarkers for ML sensors in the diagnosis of the disease, supporting edge computing on Arduino and cloud IoT service. △ Less

Submitted 20 October, 2022; v1 submitted 7 September, 2022; originally announced September 2022.

Comments: 30 pages, 9 figures, 8 tables, 1 algorithm

Journal ref: Sensors 2022, 22, 7886

Interpretable Vertebral Fracture Quantification via Anchor-Free Landmarks Localization

Authors: Alexey Zakharov, Maxim Pisov, Alim Bukharaev, Alexey Petraikin, Sergey Morozov, Victor Gombolevskiy, Mikhail Belyaev

Abstract: Vertebral body compression fractures are early signs of osteoporosis. Though these fractures are visible on Computed Tomography (CT) images, they are frequently missed by radiologists in clinical settings. Prior research on automatic methods of vertebral fracture classification proves its reliable quality; however, existing methods provide hard-to-interpret outputs and sometimes fail to process ca… ▽ More Vertebral body compression fractures are early signs of osteoporosis. Though these fractures are visible on Computed Tomography (CT) images, they are frequently missed by radiologists in clinical settings. Prior research on automatic methods of vertebral fracture classification proves its reliable quality; however, existing methods provide hard-to-interpret outputs and sometimes fail to process cases with severe abnormalities such as highly pathological vertebrae or scoliosis. We propose a new two-step algorithm to localize the vertebral column in 3D CT images and then detect individual vertebrae and quantify fractures in 2D simultaneously. We train neural networks for both steps using a simple 6-keypoints based annotation scheme, which corresponds precisely to the current clinical recommendation. Our algorithm has no exclusion criteria, processes 3D CT in 2 seconds on a single GPU, and provides an interpretable and verifiable output. The method approaches expert-level performance and demonstrates state-of-the-art results in vertebrae 3D localization (the average error is 1 mm), vertebrae 2D detection (precision and recall are 0.99), and fracture identification (ROC AUC at the patient level is up to 0.96). Our anchor-free vertebra detection network shows excellent generalizability on a new domain by achieving ROC AUC 0.95, sensitivity 0.85, specificity 0.9 on a challenging VerSe dataset with many unseen vertebra types. △ Less

Submitted 1 October, 2022; v1 submitted 14 April, 2022; originally announced April 2022.

Comments: arXiv admin note: text overlap with arXiv:2005.11960

Negligible effect of brain MRI data preprocessing for tumor segmentation

Authors: Ekaterina Kondrateva, Polina Druzhinina, Alexandra Dalechina, Svetlana Zolotova, Andrey Golanov, Boris Shirokikh, Mikhail Belyaev, Anvar Kurmukov

Abstract: Magnetic resonance imaging (MRI) data is heterogeneous due to differences in device manufacturers, scanning protocols, and inter-subject variability. A conventional way to mitigate MR image heterogeneity is to apply preprocessing transformations such as anatomy alignment, voxel resampling, signal intensity equalization, image denoising, and localization of regions of interest. Although a preproces… ▽ More Magnetic resonance imaging (MRI) data is heterogeneous due to differences in device manufacturers, scanning protocols, and inter-subject variability. A conventional way to mitigate MR image heterogeneity is to apply preprocessing transformations such as anatomy alignment, voxel resampling, signal intensity equalization, image denoising, and localization of regions of interest. Although a preprocessing pipeline standardizes image appearance, its influence on the quality of image segmentation and on other downstream tasks in deep neural networks has never been rigorously studied. We conduct experiments on three publicly available datasets and evaluate the effect of different preprocessing steps in intra- and inter-dataset training scenarios. Our results demonstrate that most popular standardization steps add no value to the network performance; moreover, preprocessing can hamper model performance. We suggest that image intensity normalization approaches do not contribute to model accuracy because of the reduction of signal variance with image standardization. Finally, we show that the contribution of skull-stripping in data preprocessing is almost negligible if measured in terms of estimated tumor volume. We show that the only essential transformation for accurate deep learning analysis is the unification of voxel spacing across the dataset. In contrast, inter-subjects anatomy alignment in the form of non-rigid atlas registration is not necessary and intensity equalization steps (denoising, bias-field correction and histogram matching) do not improve models' performance. The study code is accessible online https://github.com/MedImAIR/brain-mri-processing-pipeline △ Less

Submitted 23 October, 2023; v1 submitted 11 April, 2022; originally announced April 2022.

Adaptation to CT Reconstruction Kernels by Enforcing Cross-domain Feature Maps Consistency

Authors: Stanislav Shimovolos, Andrey Shushko, Mikhail Belyaev, Boris Shirokikh

Abstract: Deep learning methods provide significant assistance in analyzing coronavirus disease (COVID-19) in chest computed tomography (CT) images, including identification, severity assessment, and segmentation. Although the earlier developed methods address the lack of data and specific annotations, the current goal is to build a robust algorithm for clinical use, having a larger pool of available data.… ▽ More Deep learning methods provide significant assistance in analyzing coronavirus disease (COVID-19) in chest computed tomography (CT) images, including identification, severity assessment, and segmentation. Although the earlier developed methods address the lack of data and specific annotations, the current goal is to build a robust algorithm for clinical use, having a larger pool of available data. With the larger datasets, the domain shift problem arises, affecting the performance of methods on the unseen data. One of the critical sources of domain shift in CT images is the difference in reconstruction kernels used to generate images from the raw data (sinograms). In this paper, we show a decrease in the COVID-19 segmentation quality of the model trained on the smooth and tested on the sharp reconstruction kernels. Furthermore, we compare several domain adaptation approaches to tackle the problem, such as task-specific augmentation and unsupervised adversarial learning. Finally, we propose the unsupervised adaptation method, called F-Consistency, that outperforms the previous approaches. Our method exploits a set of unlabeled CT image pairs which differ only in reconstruction kernels within every pair. It enforces the similarity of the network hidden representations (feature maps) by minimizing mean squared error (MSE) between paired feature maps. We show our method achieving 0.64 Dice Score on the test dataset with unseen sharp kernels, compared to the 0.56 Dice Score of the baseline model. Moreover, F-Consistency scores 0.80 Dice Score between predictions on the paired images, which almost doubles the baseline score of 0.46 and surpasses the other methods. We also show F-Consistency to better generalize on the unseen kernels and without the specific semantic content, e.g., presence of the COVID-19 lesions. △ Less

Submitted 28 March, 2022; originally announced March 2022.

Systematic Clinical Evaluation of A Deep Learning Method for Medical Image Segmentation: Radiosurgery Application

Authors: Boris Shirokikh, Alexandra Dalechina, Alexey Shevtsov, Egor Krivov, Valery Kostjuchenko, Amayak Durgaryan, Mikhail Galkin, Andrey Golanov, Mikhail Belyaev

Abstract: We systematically evaluate a Deep Learning (DL) method in a 3D medical image segmentation task. Our segmentation method is integrated into the radiosurgery treatment process and directly impacts the clinical workflow. With our method, we address the relative drawbacks of manual segmentation: high inter-rater contouring variability and high time consumption of the contouring process. The main exten… ▽ More We systematically evaluate a Deep Learning (DL) method in a 3D medical image segmentation task. Our segmentation method is integrated into the radiosurgery treatment process and directly impacts the clinical workflow. With our method, we address the relative drawbacks of manual segmentation: high inter-rater contouring variability and high time consumption of the contouring process. The main extension over the existing evaluations is the careful and detailed analysis that could be further generalized on other medical image segmentation tasks. Firstly, we analyze the changes in the inter-rater detection agreement. We show that the segmentation model reduces the ratio of detection disagreements from 0.162 to 0.085 (p < 0.05). Secondly, we show that the model improves the inter-rater contouring agreement from 0.845 to 0.871 surface Dice Score (p < 0.05). Thirdly, we show that the model accelerates the delineation process in between 1.6 and 2.0 times (p < 0.05). Finally, we design the setup of the clinical experiment to either exclude or estimate the evaluation biases, thus preserve the significance of the results. Besides the clinical evaluation, we also summarize the intuitions and practical ideas for building an efficient DL-based model for 3D medical image segmentation. △ Less

Submitted 21 August, 2021; originally announced August 2021.

Zero-Shot Domain Adaptation in CT Segmentation by Filtered Back Projection Augmentation

Authors: Talgat Saparov, Anvar Kurmukov, Boris Shirokikh, Mikhail Belyaev

Abstract: Domain shift is one of the most salient challenges in medical computer vision. Due to immense variability in scanners' parameters and imaging protocols, even images obtained from the same person and the same scanner could differ significantly. We address variability in computed tomography (CT) images caused by different convolution kernels used in the reconstruction process, the critical domain sh… ▽ More Domain shift is one of the most salient challenges in medical computer vision. Due to immense variability in scanners' parameters and imaging protocols, even images obtained from the same person and the same scanner could differ significantly. We address variability in computed tomography (CT) images caused by different convolution kernels used in the reconstruction process, the critical domain shift factor in CT. The choice of a convolution kernel affects pixels' granularity, image smoothness, and noise level. We analyze a dataset of paired CT images, where smooth and sharp images were reconstructed from the same sinograms with different kernels, thus providing identical anatomy but different style. Though identical predictions are desired, we show that the consistency, measured as the average Dice between predictions on pairs, is just 0.54. We propose Filtered Back-Projection Augmentation (FBPAug), a simple and surprisingly efficient approach to augment CT images in sinogram space emulating reconstruction with different kernels. We apply the proposed method in a zero-shot domain adaptation setup and show that the consistency boosts from 0.54 to 0.92 outperforming other augmentation approaches. Neither specific preparation of source domain data nor target domain data is required, so our publicly released FBPAug can be used as a plug-and-play module for zero-shot domain adaptation in any CT-based task. △ Less

Submitted 8 June, 2022; v1 submitted 18 July, 2021; originally announced July 2021.

Comments: table fixed

Anatomy of Domain Shift Impact on U-Net Layers in MRI Segmentation

Authors: Ivan Zakazov, Boris Shirokikh, Alexey Chernyavskiy, Mikhail Belyaev

Abstract: Domain Adaptation (DA) methods are widely used in medical image segmentation tasks to tackle the problem of differently distributed train (source) and test (target) data. We consider the supervised DA task with a limited number of annotated samples from the target domain. It corresponds to one of the most relevant clinical setups: building a sufficiently accurate model on the minimum possible amou… ▽ More Domain Adaptation (DA) methods are widely used in medical image segmentation tasks to tackle the problem of differently distributed train (source) and test (target) data. We consider the supervised DA task with a limited number of annotated samples from the target domain. It corresponds to one of the most relevant clinical setups: building a sufficiently accurate model on the minimum possible amount of annotated data. Existing methods mostly fine-tune specific layers of the pretrained Convolutional Neural Network (CNN). However, there is no consensus on which layers are better to fine-tune, e.g. the first layers for images with low-level domain shift or the deeper layers for images with high-level domain shift. To this end, we propose SpotTUnet - a CNN architecture that automatically chooses the layers which should be optimally fine-tuned. More specifically, on the target domain, our method additionally learns the policy that indicates whether a specific layer should be fine-tuned or reused from the pretrained network. We show that our method performs at the same level as the best of the nonflexible fine-tuning methods even under the extreme scarcity of annotated data. Secondly, we show that SpotTUnet policy provides a layer-wise visualization of the domain shift impact on the network, which could be further used to develop robust domain generalization methods. In order to extensively evaluate SpotTUnet performance, we use a publicly available dataset of brain MR images (CC359), characterized by explicit domain shift. We release a reproducible experimental pipeline. △ Less

Submitted 10 July, 2021; originally announced July 2021.

Comments: Accepted for MICCAI-2021 conference

Type-Centric Kotlin Compiler Fuzzing: Preserving Test Program Correctness by Preserving Types

Authors: Daniil Stepanov, Marat Akhin, Mikhail Belyaev

Abstract: Kotlin is a relatively new programming language from JetBrains: its development started in 2010 with release 1.0 done in early 2016. The Kotlin compiler, while slowly and steadily becoming more and more mature, still crashes from time to time on the more tricky input programs, not least because of the complexity of its features and their interactions. This makes it a great target for fuzzing, even… ▽ More Kotlin is a relatively new programming language from JetBrains: its development started in 2010 with release 1.0 done in early 2016. The Kotlin compiler, while slowly and steadily becoming more and more mature, still crashes from time to time on the more tricky input programs, not least because of the complexity of its features and their interactions. This makes it a great target for fuzzing, even the basic forms of which can find a significant number of Kotlin compiler crashes. There is a problem with fuzzing, however, closely related to the cause of the crashes: generating a random, non-trivial and semantically valid Kotlin program is hard. In this paper, we talk about type-centric compiler fuzzing in the form of type-centric enumeration, an approach inspired by skeletal program enumeration and based on a combination of generative and mutation-based fuzzing, which solves this problem by focusing on program types. After creating the skeleton program, we fill the typed holes with fragments of suitable type, created via generation and enhanced by semantic-aware mutation. We implemented this approach in our Kotlin compiler fuzzing framework called Backend Bug Finder (BBF) and did an extensive evaluation, not only testing the real-world feasibility of our approach, but also comparing it to other compiler fuzzing techniques. The results show our approach to be significantly better compared to other fuzzing approaches at generating semantically valid Kotlin programs, while creating more interesting crash-inducing inputs at the same time. We managed to find more than 50 previously unknown compiler crashes, of which 18 were considered important after their triage by the compiler team. △ Less

Submitted 11 December, 2020; originally announced December 2020.

Comments: Accepted to: 2021 IEEE International Conference on Software Testing, Verification and Validation (ICST)

Electron beam modification of vanadium dioxide oscillators

Authors: Maksim Belyaev, Andrei Velichko, Vadim Putrolaynen, Valentin Perminov, Alexander Per-gament

Abstract: The paper presents the results of a study of electron-beam modification (EBM) of VO2-switch I-V curve threshold parameters and the self-oscillation frequency of a circuit containing such a switching device. EBM in vacuum is reversible and the parameters are restored when exposed to air at pressure of 150 Pa. At EBM with a dose of 3 C/cm2, the voltages of switching-on (Vth) and off (Vh), as well as… ▽ More The paper presents the results of a study of electron-beam modification (EBM) of VO2-switch I-V curve threshold parameters and the self-oscillation frequency of a circuit containing such a switching device. EBM in vacuum is reversible and the parameters are restored when exposed to air at pressure of 150 Pa. At EBM with a dose of 3 C/cm2, the voltages of switching-on (Vth) and off (Vh), as well as the OFF-state resistance Roff, decrease down to 50% of the initial values, and the oscillation frequency increases by 30% at a dose of 0.7 C/cm2. Features of physics of EBM of an oscillator are outlined considering the contribution of the metal and semiconductor phases of the switching channel. Con-trolled modification allows EBM forming of switches with preset parameters. Also, it might be used in artifi-cial oscillatory neural networks for pattern recognition based on frequency shift keying. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Journal ref: Phys. Status Solidi Curr. Top. Solid State Phys. 2017, 14

First U-Net Layers Contain More Domain Specific Information Than The Last Ones

Authors: Boris Shirokikh, Ivan Zakazov, Alexey Chernyavskiy, Irina Fedulova, Mikhail Belyaev

Abstract: MRI scans appearance significantly depends on scanning protocols and, consequently, the data-collection institution. These variations between clinical sites result in dramatic drops of CNN segmentation quality on unseen domains. Many of the recently proposed MRI domain adaptation methods operate with the last CNN layers to suppress domain shift. At the same time, the core manifestation of MRI vari… ▽ More MRI scans appearance significantly depends on scanning protocols and, consequently, the data-collection institution. These variations between clinical sites result in dramatic drops of CNN segmentation quality on unseen domains. Many of the recently proposed MRI domain adaptation methods operate with the last CNN layers to suppress domain shift. At the same time, the core manifestation of MRI variability is a considerable diversity of image intensities. We hypothesize that these differences can be eliminated by modifying the first layers rather than the last ones. To validate this simple idea, we conducted a set of experiments with brain MRI scans from six domains. Our results demonstrate that 1) domain-shift may deteriorate the quality even for a simple brain extraction segmentation task (surface Dice Score drops from 0.85-0.89 even to 0.09); 2) fine-tuning of the first layers significantly outperforms fine-tuning of the last layers in almost all supervised domain adaptation setups. Moreover, fine-tuning of the first layers is a better strategy than fine-tuning of the whole network, if the amount of annotated data from the new domain is strictly limited. △ Less

Submitted 17 August, 2020; originally announced August 2020.

Comments: Accepted to DART workshop at MICCAI-2020

Universal Loss Reweighting to Balance Lesion Size Inequality in 3D Medical Image Segmentation

Authors: Boris Shirokikh, Alexey Shevtsov, Anvar Kurmukov, Alexandra Dalechina, Egor Krivov, Valery Kostjuchenko, Andrey Golanov, Mikhail Belyaev

Abstract: Target imbalance affects the performance of recent deep learning methods in many medical image segmentation tasks. It is a twofold problem: class imbalance - positive class (lesion) size compared to negative class (non-lesion) size; lesion size imbalance - large lesions overshadows small ones (in the case of multiple lesions per image). While the former was addressed in multiple works, the latter… ▽ More Target imbalance affects the performance of recent deep learning methods in many medical image segmentation tasks. It is a twofold problem: class imbalance - positive class (lesion) size compared to negative class (non-lesion) size; lesion size imbalance - large lesions overshadows small ones (in the case of multiple lesions per image). While the former was addressed in multiple works, the latter lacks investigation. We propose a loss reweighting approach to increase the ability of the network to detect small lesions. During the learning process, we assign a weight to every image voxel. The assigned weights are inversely proportional to the lesion volume, thus smaller lesions get larger weights. We report the benefit from our method for well-known loss functions, including Dice Loss, Focal Loss, and Asymmetric Similarity Loss. Additionally, we compare our results with other reweighting techniques: Weighted Cross-Entropy and Generalized Dice Loss. Our experiments show that inverse weighting considerably increases the detection quality, while preserves the delineation quality on a state-of-the-art level. We publish a complete experimental pipeline for two publicly available datasets of CT images: LiTS and LUNA16 (https://github.com/neuro-ml/inverse_weighting). We also show results on a private database of MR images for the task of multiple brain metastases delineation. △ Less

Submitted 20 July, 2020; originally announced July 2020.

Comments: Accepted to MICCAI 2020

CT-based COVID-19 Triage: Deep Multitask Learning Improves Joint Identification and Severity Quantification

Authors: Mikhail Goncharov, Maxim Pisov, Alexey Shevtsov, Boris Shirokikh, Anvar Kurmukov, Ivan Blokhin, Valeria Chernina, Alexander Solovev, Victor Gombolevskiy, Sergey Morozov, Mikhail Belyaev

Abstract: The current COVID-19 pandemic overloads healthcare systems, including radiology departments. Though several deep learning approaches were developed to assist in CT analysis, nobody considered study triage directly as a computer science problem. We describe two basic setups: Identification of COVID-19 to prioritize studies of potentially infected patients to isolate them as early as possible; Sever… ▽ More The current COVID-19 pandemic overloads healthcare systems, including radiology departments. Though several deep learning approaches were developed to assist in CT analysis, nobody considered study triage directly as a computer science problem. We describe two basic setups: Identification of COVID-19 to prioritize studies of potentially infected patients to isolate them as early as possible; Severity quantification to highlight studies of severe patients and direct them to a hospital or provide emergency medical care. We formalize these tasks as binary classification and estimation of affected lung percentage. Though similar problems were well-studied separately, we show that existing methods provide reasonable quality only for one of these setups. We employ a multitask approach to consolidate both triage approaches and propose a convolutional neural network to combine all available labels within a single model. In contrast with the most popular multitask approaches, we add classification layers to the most spatially detailed upper part of U-Net instead of the bottom, less detailed latent representation. We train our model on approximately 2000 publicly available CT studies and test it with a carefully designed set consisting of 32 COVID-19 studies, 30 cases with bacterial pneumonia, 31 healthy patients, and 30 patients with other lung pathologies to emulate a typical patient flow in an out-patient hospital. The proposed multitask model outperforms the latent-based one and achieves ROC AUC scores ranging from 0.87+-01 (bacterial pneumonia) to 0.97+-01 (healthy controls) for Identification of COVID-19 and 0.97+-01 Spearman Correlation for Severity quantification. We release all the code and create a public leaderboard, where other community members can test their models on our test dataset. △ Less

Submitted 26 November, 2020; v1 submitted 2 June, 2020; originally announced June 2020.

Keypoints Localization for Joint Vertebra Detection and Fracture Severity Quantification

Authors: Maxim Pisov, Vladimir Kondratenko, Alexey Zakharov, Alexey Petraikin, Victor Gombolevskiy, Sergey Morozov, Mikhail Belyaev

Abstract: Vertebral body compression fractures are reliable early signs of osteoporosis. Though these fractures are visible on Computed Tomography (CT) images, they are frequently missed by radiologists in clinical settings. Prior research on automatic methods of vertebral fracture classification proves its reliable quality; however, existing methods provide hard-to-interpret outputs and sometimes fail to p… ▽ More Vertebral body compression fractures are reliable early signs of osteoporosis. Though these fractures are visible on Computed Tomography (CT) images, they are frequently missed by radiologists in clinical settings. Prior research on automatic methods of vertebral fracture classification proves its reliable quality; however, existing methods provide hard-to-interpret outputs and sometimes fail to process cases with severe abnormalities such as highly pathological vertebrae or scoliosis. We propose a new two-step algorithm to localize the vertebral column in 3D CT images and then to simultaneously detect individual vertebrae and quantify fractures in 2D. We train neural networks for both steps using a simple 6-keypoints based annotation scheme, which corresponds precisely to current medical recommendation. Our algorithm has no exclusion criteria, processes 3D CT in 2 seconds on a single GPU, and provides an intuitive and verifiable output. The method approaches expert-level performance and demonstrates state-of-the-art results in vertebrae 3D localization (the average error is 1 mm), vertebrae 2D detection (precision is 0.99, recall is 1), and fracture identification (ROC AUC at the patient level is 0.93). △ Less

Submitted 20 July, 2020; v1 submitted 25 May, 2020; originally announced May 2020.

Comments: Accepted to MICCAI-2020

Electrical switching and oscillations in vanadium dioxide

Authors: Alexander Pergament, Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen

Abstract: We have studied electrical switching with S-shaped I-V characteristics in two-terminal MOM devices based on vanadium dioxide thin films. The switching effect is associated with the metal-insulator phase transition. Relaxation oscillations are observed in circuits with VO2-based switches. Dependences of the oscillator critical frequency Fmax, threshold power and voltage, as well as the time of curr… ▽ More We have studied electrical switching with S-shaped I-V characteristics in two-terminal MOM devices based on vanadium dioxide thin films. The switching effect is associated with the metal-insulator phase transition. Relaxation oscillations are observed in circuits with VO2-based switches. Dependences of the oscillator critical frequency Fmax, threshold power and voltage, as well as the time of current rise, on the switching structure size are obtained by numerical simulation. The empirical dependence of the threshold voltage on the switching region dimensions and film thickness is found. It is shown that, for the VO2 channel sizes of 10*10 nm, Fmax can reach the value of 300 MHz at a film thickness of ~20 nm. Next, it is shown that oscillatory neural networks can be implemented on the basis of coupled VO2 oscillators. For the weak capacitive coupling, we revealed the dependence of the phase difference upon synchronization on the coupling capacitance value. When the switches are scaled down, the limiting time of synchronization is reduced to Ts ~13 μs, and the number of oscillation periods for the entering to the synchronization mode remains constant, Ns ~ 17. In the case of weak thermal coupling in the synchronization mode, we observe in-phase behavior of oscillators, and there is a certain range of parameters of the supply current, in which the synchronization effect becomes possible. With a decrease in dimensions, a decrease in the thermal coupling action radius is observed, which can vary in the range from 0.5 to 50 μm for structures with characteristic dimensions of 0.1 to 5 μm, respectively. Thermal coupling may have a promising effect for realization of a 3D integrated oscillatory neural network. △ Less

Submitted 7 January, 2020; originally announced January 2020.

Comments: 28 pages, 13 figures

Journal ref: Physica B: Condensed Matter, Volume 536, 1 May 2018, Pages 239-248

Switching dynamics of single and coupled VO2-based oscillators as elements of neural networks

Authors: Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen, Alexander Pergament, Valentin Perminov

Abstract: In the present paper, we report on the switching dynamics of both single and coupled VO2-based oscillators, with resistive and capacitive coupling, and explore the capability of their application in oscillatory neural networks. Based on these results, we further select an adequate SPICE model to describe the modes of operation of coupled oscillator circuits. Physical mechanisms influencing the tim… ▽ More In the present paper, we report on the switching dynamics of both single and coupled VO2-based oscillators, with resistive and capacitive coupling, and explore the capability of their application in oscillatory neural networks. Based on these results, we further select an adequate SPICE model to describe the modes of operation of coupled oscillator circuits. Physical mechanisms influencing the time of forward and reverse electrical switching, that determine the applicability limits of the proposed model, are identified. For the resistive coupling, it is shown that synchronization takes place at a certain value of the coupling resistance, though it is unstable and a synchronization failure occurs periodically. For the capacitive coupling, two synchronization modes, with weak and strong coupling, are found. The transition between these modes is accompanied by chaotic oscillations. A decrease in the width of the spectrum harmonics in the weak-coupling mode, and its increase in the strong-coupling one, is detected. The dependences of frequencies and phase differences of the coupled oscillatory circuits on the coupling capacitance are found. Examples of operation of coupled VO2 oscillators as a central pattern generator are demonstrated. △ Less

Submitted 6 January, 2020; originally announced January 2020.

Comments: 33 pages, 23 figures

Journal ref: International Journal of Modern Physics B, Vol. 31, No. 02, 1650261 (2017)

Thermal coupling and effect of subharmonic synchronization in a system of two VO2 based oscillators

Authors: Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen, Valentin Perminov, Alexander Pergament

Abstract: We explore a prototype of an oscillatory neural network (ONN) based on vanadium dioxide switching devices. The model system under study represents two oscillators based on thermally coupled VO2 switches. Numerical simulation shows that the effective action radius RTC of coupling depends both on the total energy released during switching and on the average power. It is experimentally and numericall… ▽ More We explore a prototype of an oscillatory neural network (ONN) based on vanadium dioxide switching devices. The model system under study represents two oscillators based on thermally coupled VO2 switches. Numerical simulation shows that the effective action radius RTC of coupling depends both on the total energy released during switching and on the average power. It is experimentally and numerically proved that the temperature change dT commences almost synchronously with the released power peak and T-coupling reveals itself up to a frequency of about 10 kHz. For the studied switching structure configuration, the RTC value varies over a wide range from 4 to 45 mkm, depending on the external circuit capacitance C and resistance Ri, but the variation of Ri is more promising from the practical viewpoint. In the case of a "weak" coupling, synchronization is accompanied by attraction effect and decrease of the main spectra harmonics width. In the case of a "strong" coupling, the number of effects increases, synchronization can occur on subharmonics resulting in multilevel stable synchronization of two oscillators. An advanced algorithm for synchronization efficiency and subharmonic ratio calculation is proposed. It is shown that of the two oscillators the leading one is that with a higher main frequency, and, in addition, the frequency stabilization effect is observed. Also, in the case of a strong thermal coupling, the limit of the supply current parameters, for which the oscillations exist, expands by ~ 10 %. The obtained results have a universal character and open up a new kind of coupling in ONNs, namely, T-coupling, which allows for easy transition from 2D to 3D integration. The effect of subharmonic synchronization hold promise for application in classification and pattern recognition. △ Less

Submitted 5 January, 2020; originally announced January 2020.

Comments: 24 pages, 10 figures

Journal ref: Solid. State. Electron. 2018, 141, 40-49

Capacitorless Model of a VO2 Oscillator

Authors: M. A. Belyaev, A. A. Velichko

Abstract: We implement a capacitorless model of a VO2 oscillator by introducing into the circuit of a field-effect transistor and a VO2 thermal sensor, which provide negative current feedback with a time delay. We compare the dynamics of current and voltage oscillations on a switch in a circuit with a capacitor and without a capacitor. The oscillation period in the capacitorless model is controlled in a nar… ▽ More We implement a capacitorless model of a VO2 oscillator by introducing into the circuit of a field-effect transistor and a VO2 thermal sensor, which provide negative current feedback with a time delay. We compare the dynamics of current and voltage oscillations on a switch in a circuit with a capacitor and without a capacitor. The oscillation period in the capacitorless model is controlled in a narrow range by changing the distance between the switch and the sensor. The capacitorless model provides the possibility of significant miniaturization of the oscillator circuit, and it is important for the implementation of large arrays of oscillators in oscillatory neural networks to solve the problem of classification and pattern recognition. △ Less

Submitted 16 November, 2019; originally announced November 2019.

Comments: 7 pages, 5 figures

Journal ref: IOP Conf. Ser.: Mater. Sci. Eng. 734 012151 (2020)

Multi-domain CT Metal Artifacts Reduction Using Partial Convolution Based Inpainting

Authors: Artem Pimkin, Alexander Samoylenko, Natalia Antipina, Anna Ovechkina, Andrey Golanov, Alexandra Dalechina, Mikhail Belyaev

Abstract: Recent CT Metal Artifacts Reduction (MAR) methods are often based on image-to-image convolutional neural networks for adjustment of corrupted sinograms or images themselves. In this paper, we are exploring the capabilities of a multi-domain method which consists of both sinogram correction (projection domain step) and restored image correction (image-domain step). Moreover, we propose a formulatio… ▽ More Recent CT Metal Artifacts Reduction (MAR) methods are often based on image-to-image convolutional neural networks for adjustment of corrupted sinograms or images themselves. In this paper, we are exploring the capabilities of a multi-domain method which consists of both sinogram correction (projection domain step) and restored image correction (image-domain step). Moreover, we propose a formulation of the first step problem as sinogram inpainting which allows us to use methods of this specific field such as partial convolutions. The proposed method allows to achieve state-of-the-art (-75% MSE) improvement in comparison with a classic benchmark - Li-MAR. △ Less

Submitted 11 May, 2020; v1 submitted 13 November, 2019; originally announced November 2019.

The non-capacitor model of leaky integrate-and-fire $VO_2$ neuron with the thermal mechanism of the membrane potential

Authors: A. A. Velichko, M. A. Belyaev, D. V. Ryabokon, S. D. Khanin

Abstract: The study presents a numerical model of leaky integrate-and-fire neuron created on the basis of $VO_2$ switch. The analogue of the membrane potential in the model is the temperature of the switch channel, and the action potential from neighbouring neurons propagates along the substrate in the form of thermal pulses. We simulated the operation of three neurons and demonstrated that the total effect… ▽ More The study presents a numerical model of leaky integrate-and-fire neuron created on the basis of $VO_2$ switch. The analogue of the membrane potential in the model is the temperature of the switch channel, and the action potential from neighbouring neurons propagates along the substrate in the form of thermal pulses. We simulated the operation of three neurons and demonstrated that the total effect happens due to interference of thermal waves in the region of the neuron switching channel. The thermal mechanism of the threshold function operates due to the effect of electrical switching, and the magnitude (temperature) of the threshold can vary by external voltage. The neuron circuit does not contain capacitor, making it possible to produce a network with a high density of components, and has the potential for 3D integration due to the thermal mechanism of neurons interaction. △ Less

Submitted 7 October, 2019; originally announced November 2019.

MSC Class: 68T10 ACM Class: I.5.5

Journal ref: J. Phys.: Conf. Ser. 1399 022046 (2019)

ReduKtor: How We Stopped Worrying About Bugs in Kotlin Compiler

Authors: Daniil Stepanov, Marat Akhin, Mikhail Belyaev

Abstract: Bug localization is well-known to be a difficult problem in software engineering, and specifically in compiler development, where it is beneficial to reduce the input program to a minimal reproducing example; this technique is more commonly known as delta debugging. What additionally contributes to the problem is that every new programming language has its own unique quirks and foibles, making it… ▽ More Bug localization is well-known to be a difficult problem in software engineering, and specifically in compiler development, where it is beneficial to reduce the input program to a minimal reproducing example; this technique is more commonly known as delta debugging. What additionally contributes to the problem is that every new programming language has its own unique quirks and foibles, making it near impossible to reuse existing tools and approaches with full efficiency. In this experience paper we tackle the delta debugging problem w.r.t. Kotlin, a relatively new programming language from JetBrains. Our approach is based on a novel combination of program slicing, hierarchical delta debugging and Kotlin-specific transformations, which are synergistic to each other. We implemented it in a prototype called ReduKtor and did extensive evaluation on both synthetic and real Kotlin programs; we also compared its performance with classic delta debugging techniques. The evaluation results support the practical usability of our approach to Kotlin delta debugging and also shows the importance of using both language-agnostic and language-specific techniques to achieve best reduction efficiency and performance. △ Less

Submitted 16 September, 2019; originally announced September 2019.

Comments: Accepted to: 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)

Deep Learning for Brain Tumor Segmentation in Radiosurgery: Prospective Clinical Evaluation

Authors: Boris Shirokikh, Alexandra Dalechina, Alexey Shevtsov, Egor Krivov, Valery Kostjuchenko, Amayak Durgaryan, Mikhail Galkin, Ivan Osinov, Andrey Golanov, Mikhail Belyaev

Abstract: Stereotactic radiosurgery is a minimally-invasive treatment option for a large number of patients with intracranial tumors. As part of the therapy treatment, accurate delineation of brain tumors is of great importance. However, slice-by-slice manual segmentation on T1c MRI could be time-consuming (especially for multiple metastases) and subjective. In our work, we compared several deep convolution… ▽ More Stereotactic radiosurgery is a minimally-invasive treatment option for a large number of patients with intracranial tumors. As part of the therapy treatment, accurate delineation of brain tumors is of great importance. However, slice-by-slice manual segmentation on T1c MRI could be time-consuming (especially for multiple metastases) and subjective. In our work, we compared several deep convolutional networks architectures and training procedures and evaluated the best model in a radiation therapy department for three types of brain tumors: meningiomas, schwannomas and multiple brain metastases. The developed semiautomatic segmentation system accelerates the contouring process by 2.2 times on average and increases inter-rater agreement from 92.0% to 96.5%. △ Less

Submitted 18 December, 2019; v1 submitted 6 September, 2019; originally announced September 2019.

Incorporating Task-Specific Structural Knowledge into CNNs for Brain Midline Shift Detection

Authors: Maxim Pisov, Mikhail Goncharov, Nadezhda Kurochkina, Sergey Morozov, Victor Gombolevskiy, Valeria Chernina, Anton Vladzymyrskyy, Ksenia Zamyatina, Anna Chesnokova, Igor Pronin, Michael Shifrin, Mikhail Belyaev

Abstract: Midline shift (MLS) is a well-established factor used for outcome prediction in traumatic brain injury, stroke and brain tumors. The importance of automatic estimation of MLS was recently highlighted by ACR Data Science Institute. In this paper we introduce a novel deep learning based approach for the problem of MLS detection, which exploits task-specific structural knowledge. We evaluate our meth… ▽ More Midline shift (MLS) is a well-established factor used for outcome prediction in traumatic brain injury, stroke and brain tumors. The importance of automatic estimation of MLS was recently highlighted by ACR Data Science Institute. In this paper we introduce a novel deep learning based approach for the problem of MLS detection, which exploits task-specific structural knowledge. We evaluate our method on a large dataset containing heterogeneous images with significant MLS and show that its mean error approaches the inter-expert variability. Finally, we show the robustness of our approach by validating it on an external dataset, acquired during routine clinical practice. △ Less

Submitted 14 December, 2019; v1 submitted 13 August, 2019; originally announced August 2019.

Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge

Authors: Hugo J. Kuijf, J. Matthijs Biesbroek, Jeroen de Bresser, Rutger Heinen, Simon Andermatt, Mariana Bento, Matt Berseth, Mikhail Belyaev, M. Jorge Cardoso, Adrià Casamitjana, D. Louis Collins, Mahsa Dadar, Achilleas Georgiou, Mohsen Ghafoorian, Dakai Jin, April Khademi, Jesse Knight, Hongwei Li, Xavier Lladó, Miguel Luna, Qaiser Mahmood, Richard McKinley, Alireza Mehrtash, Sébastien Ourselin, Bo-yong Park , et al. (19 additional authors not shown)

Abstract: Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. Automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We… ▽ More Quantification of cerebral white matter hyperintensities (WMH) of presumed vascular origin is of key importance in many neurological research studies. Currently, measurements are often still obtained from manual segmentations on brain MR images, which is a laborious procedure. Automatic WMH segmentation methods exist, but a standardized comparison of the performance of such methods is lacking. We organized a scientific challenge, in which developers could evaluate their method on a standardized multi-center/-scanner image dataset, giving an objective comparison: the WMH Segmentation Challenge (https://wmh.isi.uu.nl/). Sixty T1+FLAIR images from three MR scanners were released with manual WMH segmentations for training. A test set of 110 images from five MR scanners was used for evaluation. Segmentation methods had to be containerized and submitted to the challenge organizers. Five evaluation metrics were used to rank the methods: (1) Dice similarity coefficient, (2) modified Hausdorff distance (95th percentile), (3) absolute log-transformed volume difference, (4) sensitivity for detecting individual lesions, and (5) F1-score for individual lesions. Additionally, methods were ranked on their inter-scanner robustness. Twenty participants submitted their method for evaluation. This paper provides a detailed analysis of the results. In brief, there is a cluster of four methods that rank significantly better than the other methods, with one clear winner. The inter-scanner robustness ranking shows that not all methods generalize to unseen scanners. The challenge remains open for future submissions and provides a public platform for method evaluation. △ Less

Submitted 1 April, 2019; originally announced April 2019.

Comments: Accepted for publication in IEEE Transactions on Medical Imaging

Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge

Authors: Spyridon Bakas, Mauricio Reyes, Andras Jakab, Stefan Bauer, Markus Rempfler, Alessandro Crimi, Russell Takeshi Shinohara, Christoph Berger, Sung Min Ha, Martin Rozycki, Marcel Prastawa, Esther Alberts, Jana Lipkova, John Freymann, Justin Kirby, Michel Bilello, Hassan Fathallah-Shaykh, Roland Wiest, Jan Kirschke, Benedikt Wiestler, Rivka Colen, Aikaterini Kotrotsou, Pamela Lamontagne, Daniel Marcus, Mikhail Milchenko , et al. (402 additional authors not shown)

Abstract: Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles dissem… ▽ More Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset. △ Less

Submitted 23 April, 2019; v1 submitted 5 November, 2018; originally announced November 2018.

Comments: The International Multimodal Brain Tumor Segmentation (BraTS) Challenge

Brain Tumor Image Retrieval via Multitask Learning

Authors: Maxim Pisov, Gleb Makarchuk, Valery Kostjuchenko, Alexandra Dalechina, Andrey Golanov, Mikhail Belyaev

Abstract: Classification-based image retrieval systems are built by training convolutional neural networks (CNNs) on a relevant classification problem and using the distance in the resulting feature space as a similarity metric. However, in practical applications, it is often desirable to have representations which take into account several aspects of the data (e.g., brain tumor type and its localization).… ▽ More Classification-based image retrieval systems are built by training convolutional neural networks (CNNs) on a relevant classification problem and using the distance in the resulting feature space as a similarity metric. However, in practical applications, it is often desirable to have representations which take into account several aspects of the data (e.g., brain tumor type and its localization). In our work, we extend the classification-based approach with multitask learning: we train a CNN on brain MRI scans with heterogeneous labels and implement a corresponding tumor image retrieval system. We validate our approach on brain tumor data which contains information about tumor types, shapes and localization. We show that our method allows us to build representations that contain more relevant information about tumors than single-task classification-based approaches. △ Less

Submitted 22 October, 2018; originally announced October 2018.

Tumor Delineation For Brain Radiosurgery by a ConvNet and Non-Uniform Patch Generation

Authors: Egor Krivov, Valery Kostjuchenko, Alexandra Dalechina, Boris Shirokikh, Gleb karchuk, Alexander Denisenko, Andrey Golanov, Mikhail Belyaev

Abstract: Deep learning methods are actively used for brain lesion segmentation. One of the most popular models is DeepMedic, which was developed for segmentation of relatively large lesions like glioma and ischemic stroke. In our work, we consider segmentation of brain tumors appropriate to stereotactic radiosurgery which limits typical lesion sizes. These differences in target volumes lead to a large numb… ▽ More Deep learning methods are actively used for brain lesion segmentation. One of the most popular models is DeepMedic, which was developed for segmentation of relatively large lesions like glioma and ischemic stroke. In our work, we consider segmentation of brain tumors appropriate to stereotactic radiosurgery which limits typical lesion sizes. These differences in target volumes lead to a large number of false negatives (especially for small lesions) as well as to an increased number of false positives for DeepMedic. We propose a new patch-sampling procedure to increase network performance for small lesions. We used a 6-year dataset from a stereotactic radiosurgery center. To evaluate our approach, we conducted experiments with the three most frequent brain tumors: metastasis, meningioma, schwannoma. In addition to cross-validation, we estimated quality on a hold-out test set which was collected several years later than the train one. The experimental results show solid improvements in both cases. △ Less

Submitted 1 August, 2018; originally announced August 2018.

Predicting Conversion of Mild Cognitive Impairments to Alzheimer's Disease and Exploring Impact of Neuroimaging

Authors: Yaroslav Shmulev, Mikhail Belyaev

Abstract: Nowadays, a lot of scientific efforts are concentrated on the diagnosis of Alzheimer's Disease (AD) applying deep learning methods to neuroimaging data. Even for 2017, there were published more than a hundred papers dedicated to AD diagnosis, whereas only a few works considered a problem of mild cognitive impairments (MCI) conversion to the AD. However, the conversion prediction is an important pr… ▽ More Nowadays, a lot of scientific efforts are concentrated on the diagnosis of Alzheimer's Disease (AD) applying deep learning methods to neuroimaging data. Even for 2017, there were published more than a hundred papers dedicated to AD diagnosis, whereas only a few works considered a problem of mild cognitive impairments (MCI) conversion to the AD. However, the conversion prediction is an important problem since approximately 15% of patients with MCI converges to the AD every year. In the current work, we are focusing on the conversion prediction using brain Magnetic Resonance Imaging and clinical data, such as demographics, cognitive assessments, genetic, and biochemical markers. First of all, we applied state-of-the-art deep learning algorithms on the neuroimaging data and compared these results with two machine learning algorithms that we fit using the clinical data. As a result, the models trained on the clinical data outperform the deep learning algorithms applied to the MR images. To explore the impact of neuroimaging further, we trained a deep feed-forward embedding using similarity learning with Histogram loss on all available MRIs and obtained 64-dimensional vector representation of neuroimaging data. The use of learned representation from the deep embedding allowed to increase the quality of prediction based on the neuroimaging. Finally, the current results on this dataset show that the neuroimaging does affect conversion prediction, however, cannot noticeably increase the quality of the prediction. The best results of predicting MCI-to-AD conversion are provided by XGBoost algorithm trained on the clinical and embedding data. The resulting accuracy is 0.76 +- 0.01 and the area under the ROC curve - 0.86 +- 0.01. △ Less

Submitted 30 July, 2018; originally announced July 2018.

A Model of an Oscillatory Neural Network with Multilevel Neurons for Pattern Recognition and Computing

Authors: Andrei Velichko, Maksim Belyaev, Petr Boriskov

Abstract: The current study uses a novel method of multilevel neurons and high order synchronization effects described by a family of special metrics, for pattern recognition in an oscillatory neural network (ONN). The output oscillator (neuron) of the network has multilevel variations in its synchronization value with the reference oscillator, and allows classification of an input pattern into a set of cla… ▽ More The current study uses a novel method of multilevel neurons and high order synchronization effects described by a family of special metrics, for pattern recognition in an oscillatory neural network (ONN). The output oscillator (neuron) of the network has multilevel variations in its synchronization value with the reference oscillator, and allows classification of an input pattern into a set of classes. The ONN model is implemented on thermally-coupled vanadium dioxide oscillators. The ONN is trained by the simulated annealing algorithm for selection of the network parameters. The results demonstrate that ONN is capable of classifying 512 visual patterns (as a cell array 3 * 3, distributed by symmetry into 102 classes) into a set of classes with a maximum number of elements up to fourteen. The classification capability of the network depends on the interior noise level and synchronization effectiveness parameter. The model allows for designing multilevel output cascades of neural networks with high net data throughput. The presented method can be applied in ONNs with various coupling mechanisms and oscillator topology. △ Less

Submitted 23 August, 2019; v1 submitted 8 June, 2018; originally announced June 2018.

Comments: 26 pages, 24 figures

MSC Class: 68T10 ACM Class: I.5.5

Journal ref: Electronics 2019, 8(1), 75

Method of increasing the information capacity of associative memory of oscillator neural networks using high-order synchronization effect

Authors: Andrei Velichko, Maksim Belyaev, Vadim Putrolaynen, Petr Boriskov

Abstract: Computational modelling of two- and three-oscillator schemes with thermally coupled $VO_2$-switches is used to demonstrate a novel method of pattern storage and recognition in an impulse oscillator neural network (ONN) based on the high-order synchronization effect. The method ensures high information capacity of associative memory, i.e. a large number of synchronous states $N_s$. Each state in th… ▽ More Computational modelling of two- and three-oscillator schemes with thermally coupled $VO_2$-switches is used to demonstrate a novel method of pattern storage and recognition in an impulse oscillator neural network (ONN) based on the high-order synchronization effect. The method ensures high information capacity of associative memory, i.e. a large number of synchronous states $N_s$. Each state in the system is characterized by the synchronization order determined as the ratio of harmonics number at the common synchronization frequency. The modelling demonstrates attainment of $N_s$ of several orders both for a three-oscillator scheme $N_s$~650 and for a two-oscillator scheme $N_s$~260. A number of regularities are obtained, in particular, an optimal strength of oscillator coupling is revealed when $N_s$ has a maximum. A general tendency toward information capacity decrease is shown when the coupling strength and switch inner noise amplitude increase. An algorithm of pattern storage and test vector recognition is suggested. It is also shown that the coordinate number in each vector should be one less than the switch number to reduce recognition ambiguity. The demonstrated method of associative memory realization is a general one and it may be applied in ONNs with various mechanisms and oscillator coupling topology. △ Less

Submitted 14 May, 2018; originally announced May 2018.

Comments: 18 pages, 8 figures

MSC Class: 68T10 ACM Class: I.5.5

Higher Order and Long-Range Synchronization Effects for Classification and Computing in Oscillator-Based Spiking Neural Networks

Authors: Andrei Velichko, Vadim Putrolaynen, Maksim Belyaev

Abstract: In the circuit of two thermally coupled VO2 oscillators, we studied a higher order synchronization effect, which can be used in object classification techniques to increase the number of possible synchronous states of the oscillator system. We developed the phase-locking estimation method to determine the values of subharmonic ratio and synchronization effectiveness. In our experiment, the number… ▽ More In the circuit of two thermally coupled VO2 oscillators, we studied a higher order synchronization effect, which can be used in object classification techniques to increase the number of possible synchronous states of the oscillator system. We developed the phase-locking estimation method to determine the values of subharmonic ratio and synchronization effectiveness. In our experiment, the number of possible synchronous states of the oscillator system was twelve, and subharmonic ratio distributions were shaped as Arnold's tongues. In the model, the number of states may reach the maximum value of 150 at certain levels of coupling strength and noise. The long-range synchronization effect in a one-dimensional chain of oscillators occurs even at low values of synchronization effectiveness for intermediate links. We demonstrate a technique for storing and recognizing vector images, which can used for reservoir computing. In addition, we present the implementation of analog operation of multiplication, the synchronization based logic for binary computations, and the possibility to develop the interface between spike neural network and a computer. Based on the universal physical effects, the high order synchronization can be applied to any spiking oscillators with any coupling type, enhancing the practical value of the presented results to expand spike neural network capabilities. △ Less

Submitted 29 July, 2020; v1 submitted 10 April, 2018; originally announced April 2018.

Comments: 25 pages, 13 figures

Journal ref: Neural Comput. Appl. 2020

Ensembling Neural Networks for Digital Pathology Images Classification and Segmentation

Authors: Gleb Makarchuk, Vladimir Kondratenko, Maxim Pisov, Artem Pimkin, Egor Krivov, Mikhail Belyaev

Abstract: In the last years, neural networks have proven to be a powerful framework for various image analysis problems. However, some application domains have specific limitations. Notably, digital pathology is an example of such fields due to tremendous image sizes and quite limited number of training examples available. In this paper, we adopt state-of-the-art convolutional neural networks (CNN) architec… ▽ More In the last years, neural networks have proven to be a powerful framework for various image analysis problems. However, some application domains have specific limitations. Notably, digital pathology is an example of such fields due to tremendous image sizes and quite limited number of training examples available. In this paper, we adopt state-of-the-art convolutional neural networks (CNN) architectures for digital pathology images analysis. We propose to classify image patches to increase effective sample size and then to apply an ensembling technique to build prediction for the original images. To validate the developed approaches, we conducted experiments with \textit{Breast Cancer Histology Challenge} dataset and obtained 90\% accuracy for the 4-class tissue classification task. △ Less

Submitted 3 February, 2018; originally announced February 2018.