-
Implicit Neural Representations for Speed-of-Sound Estimation in Ultrasound
Authors:
Michal Byra,
Piotr Jarosik,
Piotr Karwat,
Ziemowit Klimonda,
Marcin Lewandowski
Abstract:
Accurate estimation of the speed-of-sound (SoS) is important for ultrasound (US) image reconstruction techniques and tissue characterization. Various approaches have been proposed to calculate SoS, ranging from tomography-inspired algorithms like CUTE to convolutional networks, and more recently, physics-informed optimization frameworks based on differentiable beamforming. In this work, we utilize…
▽ More
Accurate estimation of the speed-of-sound (SoS) is important for ultrasound (US) image reconstruction techniques and tissue characterization. Various approaches have been proposed to calculate SoS, ranging from tomography-inspired algorithms like CUTE to convolutional networks, and more recently, physics-informed optimization frameworks based on differentiable beamforming. In this work, we utilize implicit neural representations (INRs) for SoS estimation in US. INRs are a type of neural network architecture that encodes continuous functions, such as images or physical quantities, through the weights of a network. Implicit networks may overcome the current limitations of SoS estimation techniques, which mainly arise from the use of non-adaptable and oversimplified physical models of tissue. Moreover, convolutional networks for SoS estimation, usually trained using simulated data, often fail when applied to real tissues due to out-of-distribution and data-shift issues. In contrast, implicit networks do not require extensive training datasets since each implicit network is optimized for an individual data case. This adaptability makes them suitable for processing US data collected from varied tissues and across different imaging protocols.
We evaluated the proposed SoS estimation method based on INRs using data collected from a tissue-mimicking phantom containing four cylindrical inclusions, with SoS values ranging from 1480 m/s to 1600 m/s. The inclusions were immersed in a material with an SoS value of 1540 m/s. In experiments, the proposed method achieved strong performance, clearly demonstrating the usefulness of implicit networks for quantitative US applications.
△ Less
Submitted 21 September, 2024;
originally announced September 2024.
-
Deep meta-learning for the selection of accurate ultrasound based breast mass classifier
Authors:
Michal Byra,
Piotr Karwat,
Ivan Ryzhankow,
Piotr Komorowski,
Ziemowit Klimonda,
Lukasz Fura,
Anna Pawlowska,
Norbert Zolek,
Jerzy Litniewski
Abstract:
Standard classification methods based on handcrafted morphological and texture features have achieved good performance in breast mass differentiation in ultrasound (US). In comparison to deep neural networks, commonly perceived as "black-box" models, classical techniques are based on features that have well-understood medical and physical interpretation. However, classifiers based on morphological…
▽ More
Standard classification methods based on handcrafted morphological and texture features have achieved good performance in breast mass differentiation in ultrasound (US). In comparison to deep neural networks, commonly perceived as "black-box" models, classical techniques are based on features that have well-understood medical and physical interpretation. However, classifiers based on morphological features commonly underperform in the presence of the shadowing artifact and ill-defined mass borders, while texture based classifiers may fail when the US image is too noisy. Therefore, in practice it would be beneficial to select the classification method based on the appearance of the particular US image. In this work, we develop a deep meta-network that can automatically process input breast mass US images and recommend whether to apply the shape or texture based classifier for the breast mass differentiation. Our preliminary results demonstrate that meta-learning techniques can be used to improve the performance of the standard classifiers based on handcrafted features. With the proposed meta-learning based approach, we achieved the area under the receiver operating characteristic curve of 0.95 and accuracy of 0.91.
△ Less
Submitted 3 November, 2022;
originally announced November 2022.
-
Parameter estimation of the homodyned K distribution based on neural networks and trainable fractional-order moments
Authors:
Michal Byra,
Ziemowit Klimonda,
Piotr Jarosik
Abstract:
Homodyned K (HK) distribution has been widely used to describe the scattering phenomena arising in various research fields, such as ultrasound imaging or optics. In this work, we propose a machine learning based approach to the estimation of the HK distribution parameters. We develop neural networks that can estimate the HK distribution parameters based on the signal-to-noise ratio, skewness and k…
▽ More
Homodyned K (HK) distribution has been widely used to describe the scattering phenomena arising in various research fields, such as ultrasound imaging or optics. In this work, we propose a machine learning based approach to the estimation of the HK distribution parameters. We develop neural networks that can estimate the HK distribution parameters based on the signal-to-noise ratio, skewness and kurtosis calculated using fractional-order moments. Compared to the previous approaches, we consider the orders of the moments as trainable variables that can be optimized along with the network weights using the back-propagation algorithm. Networks are trained based on samples generated from the HK distribution. Obtained results demonstrate that the proposed method can be used to accurately estimate the HK distribution parameters.
△ Less
Submitted 16 December, 2022; v1 submitted 11 October, 2022;
originally announced October 2022.
-
Estimating the ultrasound attenuation coefficient using convolutional neural networks -- a feasibility study
Authors:
Piotr Jarosik,
Michal Byra,
Marcin Lewandowski,
Ziemowit Klimonda
Abstract:
Attenuation coefficient (AC) is a fundamental measure of tissue acoustical properties, which can be used in medical diagnostics. In this work, we investigate the feasibility of using convolutional neural networks (CNNs) to directly estimate AC from radio-frequency (RF) ultrasound signals. To develop the CNNs we used RF signals collected from tissue mimicking numerical phantoms for the AC values in…
▽ More
Attenuation coefficient (AC) is a fundamental measure of tissue acoustical properties, which can be used in medical diagnostics. In this work, we investigate the feasibility of using convolutional neural networks (CNNs) to directly estimate AC from radio-frequency (RF) ultrasound signals. To develop the CNNs we used RF signals collected from tissue mimicking numerical phantoms for the AC values in a range from 0.1 to 1.5 dB/(MHz*cm). The models were trained based on 1-D patches of RF data. We obtained mean absolute AC estimation errors of 0.08, 0.12, 0.20, 0.25 for the patch lengths: 10 mm, 5 mm, 2 mm and 1 mm, respectively. We explain the performance of the model by visualizing the frequency content associated with convolutional filters. Our study presents that the AC can be calculated using deep learning, and the weights of the CNNs can have physical interpretation.
△ Less
Submitted 19 May, 2022;
originally announced May 2022.
-
Adversarial attacks on deep learning models for fatty liver disease classification by modification of ultrasound image reconstruction method
Authors:
Michal Byra,
Grzegorz Styczynski,
Cezary Szmigielski,
Piotr Kalinowski,
Lukasz Michalowski,
Rafal Paluszkiewicz,
Bogna Ziarkiewicz-Wroblewska,
Krzysztof Zieniewicz,
Andrzej Nowicki
Abstract:
Convolutional neural networks (CNNs) have achieved remarkable success in medical image analysis tasks. In ultrasound (US) imaging, CNNs have been applied to object classification, image reconstruction and tissue characterization. However, CNNs can be vulnerable to adversarial attacks, even small perturbations applied to input data may significantly affect model performance and result in wrong outp…
▽ More
Convolutional neural networks (CNNs) have achieved remarkable success in medical image analysis tasks. In ultrasound (US) imaging, CNNs have been applied to object classification, image reconstruction and tissue characterization. However, CNNs can be vulnerable to adversarial attacks, even small perturbations applied to input data may significantly affect model performance and result in wrong output. In this work, we devise a novel adversarial attack, specific to ultrasound (US) imaging. US images are reconstructed based on radio-frequency signals. Since the appearance of US images depends on the applied image reconstruction method, we explore the possibility of fooling deep learning model by perturbing US B-mode image reconstruction method. We apply zeroth order optimization to find small perturbations of image reconstruction parameters, related to attenuation compensation and amplitude compression, which can result in wrong output. We illustrate our approach using a deep learning model developed for fatty liver disease diagnosis, where the proposed adversarial attack achieved success rate of 48%.
△ Less
Submitted 7 September, 2020;
originally announced September 2020.
-
Breast mass segmentation based on ultrasonic entropy maps and attention gated U-Net
Authors:
Michal Byra,
Piotr Jarosik,
Katarzyna Dobruch-Sobczak,
Ziemowit Klimonda,
Hanna Piotrzkowska-Wroblewska,
Jerzy Litniewski,
Andrzej Nowicki
Abstract:
We propose a novel deep learning based approach to breast mass segmentation in ultrasound (US) imaging. In comparison to commonly applied segmentation methods, which use US images, our approach is based on quantitative entropy parametric maps. To segment the breast masses we utilized an attention gated U-Net convolutional neural network. US images and entropy maps were generated based on raw US si…
▽ More
We propose a novel deep learning based approach to breast mass segmentation in ultrasound (US) imaging. In comparison to commonly applied segmentation methods, which use US images, our approach is based on quantitative entropy parametric maps. To segment the breast masses we utilized an attention gated U-Net convolutional neural network. US images and entropy maps were generated based on raw US signals collected from 269 breast masses. The segmentation networks were developed separately using US image and entropy maps, and evaluated on a test set of 81 breast masses. The attention U-Net trained based on entropy maps achieved average Dice score of 0.60 (median 0.71), while for the model trained using US images we obtained average Dice score of 0.53 (median 0.59). Our work presents the feasibility of using quantitative US parametric maps for the breast mass segmentation. The obtained results suggest that US parametric maps, which provide the information about local tissue scattering properties, might be more suitable for the development of breast mass segmentation methods than regular US images.
△ Less
Submitted 27 January, 2020;
originally announced January 2020.
-
Knee menisci segmentation and relaxometry of 3D ultrashort echo time (UTE) cones MR imaging using attention U-Net with transfer learning
Authors:
Michal Byra,
Mei Wu,
Xiaodong Zhang,
Hyungseok Jang,
Ya-Jun Ma,
Eric Y Chang,
Sameer Shah,
Jiang Du
Abstract:
The purpose of this work is to develop a deep learning-based method for knee menisci segmentation in 3D ultrashort echo time (UTE) cones magnetic resonance (MR) imaging, and to automatically determine MR relaxation times, namely the T1, T1$ρ$, and T2* parameters, which can be used to assess knee osteoarthritis (OA). Whole knee joint imaging was performed using 3D UTE cones sequences to collect dat…
▽ More
The purpose of this work is to develop a deep learning-based method for knee menisci segmentation in 3D ultrashort echo time (UTE) cones magnetic resonance (MR) imaging, and to automatically determine MR relaxation times, namely the T1, T1$ρ$, and T2* parameters, which can be used to assess knee osteoarthritis (OA). Whole knee joint imaging was performed using 3D UTE cones sequences to collect data from 61 human subjects. Regions of interest (ROIs) were outlined by two experienced radiologists based on subtracted T1$ρ$-weighted MR images. Transfer learning was applied to develop 2D attention U-Net convolutional neural networks for the menisci segmentation based on each radiologist's ROIs separately. Dice scores were calculated to assess segmentation performance. Next, the T1, T1$ρ$, T2* relaxations, and ROI areas were determined for the manual and automatic segmentations, then compared.The models developed using ROIs provided by two radiologists achieved high Dice scores of 0.860 and 0.833, while the radiologists' manual segmentations achieved a Dice score of 0.820. Linear correlation coefficients for the T1, T1$ρ$, and T2* relaxations calculated using the automatic and manual segmentations ranged between 0.90 and 0.97, and there were no associated differences between the estimated average meniscal relaxation parameters. The deep learning models achieved segmentation performance equivalent to the inter-observer variability of two radiologists. The proposed deep learning-based approach can be used to efficiently generate automatic segmentations and determine meniscal relaxations times. The method has the potential to help radiologists with the assessment of meniscal diseases, such as OA.
△ Less
Submitted 5 August, 2019;
originally announced August 2019.
-
Breast mass classification in ultrasound based on Kendall's shape manifold
Authors:
Michal Byra,
Michael Andre
Abstract:
Morphological features play an important role in breast mass classification in sonography. While benign breast masses tend to have a well-defined ellipsoidal contour, shape of malignant breast masses is commonly ill-defined and highly variable. Various handcrafted morphological features have been developed over the years to assess this phenomenon and help the radiologists differentiate benign and…
▽ More
Morphological features play an important role in breast mass classification in sonography. While benign breast masses tend to have a well-defined ellipsoidal contour, shape of malignant breast masses is commonly ill-defined and highly variable. Various handcrafted morphological features have been developed over the years to assess this phenomenon and help the radiologists differentiate benign and malignant masses. In this paper we propose an automatic approach to morphology analysis, we express shapes of breast masses as points on the Kendall's shape manifold. Next, we use the full Procrustes distance to develop support vector machine classifiers for breast mass differentiation. The usefulness of our method is demonstrated using a dataset of B-mode images collected from 163 breast masses. Our method achieved area under the receiver operating characteristic curve of 0.81. The proposed method can be used to assess shapes of breast masses in ultrasound without any feature engineering.
△ Less
Submitted 27 May, 2019;
originally announced May 2019.
-
Quantitative Ultrasound and B-mode Image Texture Features Correlate with Collagen and Myelin Content in Human Ulnar Nerve Fascicles
Authors:
Michal Byra,
Lidi Wan,
Jonathan Wong,
Jiang Du,
Sameer Shah,
Michael Andre,
Eric Chang
Abstract:
We investigate the usefulness of quantitative ultrasound (QUS) and B-mode texture features for characterization of ulnar nerve fascicles. Ultrasound data were acquired from cadaveric specimens using a nominal 30 MHz probe. Next, the nerves were extracted to prepare histology sections. 85 fascicles were matched between the B-mode images and the histology sections. For each fascicle image, we select…
▽ More
We investigate the usefulness of quantitative ultrasound (QUS) and B-mode texture features for characterization of ulnar nerve fascicles. Ultrasound data were acquired from cadaveric specimens using a nominal 30 MHz probe. Next, the nerves were extracted to prepare histology sections. 85 fascicles were matched between the B-mode images and the histology sections. For each fascicle image, we selected an intra-fascicular region of interest. We used histology sections to determine features related to the concentration of collagen and myelin, and ultrasound data to calculate backscatter coefficient (-24.89 dB $\pm$ 8.31), attenuation coefficient (0.92 db/cm-MHz $\pm$ 0.04), Nakagami parameter (1.01 $\pm$ 0.18) and entropy (6.92 $\pm$ 0.83), as well as B-mode texture features obtained via the gray level co-occurrence matrix algorithm. Significant Spearman's rank correlations between the combined collagen and myelin concentrations were obtained for the backscatter coefficient (R=-0.68), entropy (R=-0.51), and for several texture features. Our study demonstrates that QUS may potentially provide information on structural components of nerve fascicles.
△ Less
Submitted 15 March, 2019;
originally announced March 2019.
-
Impact of ultrasound image reconstruction method on breast lesion classification with neural transfer learning
Authors:
Michal Byra,
Tomasz Sznajder,
Danijel Korzinek,
Hanna Piotrzkowska-Wroblewska,
Katarzyna Dobruch-Sobczak,
Andrzej Nowicki,
Krzysztof Marasek
Abstract:
Deep learning algorithms, especially convolutional neural networks, have become a methodology of choice in medical image analysis. However, recent studies in computer vision show that even a small modification of input image intensities may cause a deep learning model to classify the image differently. In medical imaging, the distribution of image intensities is related to applied image reconstruc…
▽ More
Deep learning algorithms, especially convolutional neural networks, have become a methodology of choice in medical image analysis. However, recent studies in computer vision show that even a small modification of input image intensities may cause a deep learning model to classify the image differently. In medical imaging, the distribution of image intensities is related to applied image reconstruction algorithm. In this paper we investigate the impact of ultrasound image reconstruction method on breast lesion classification with neural transfer learning. Due to high dynamic range raw ultrasonic signals are commonly compressed in order to reconstruct B-mode images. Based on raw data acquired from breast lesions, we reconstruct B-mode images using different compression levels. Next, transfer learning is applied for classification. Differently reconstructed images are employed for training and evaluation. We show that the modification of the reconstruction algorithm leads to decrease of classification performance. As a remedy, we propose a method of data augmentation. We show that the augmentation of the training set with differently reconstructed B-mode images leads to a more robust and efficient classification. Our study suggests that it is important to take into account image reconstruction algorithms implemented in medical scanners during development of computer aided diagnosis systems.
△ Less
Submitted 5 April, 2018;
originally announced April 2018.
