-
Nonlinear ill-posed problem in low-dose dental cone-beam computed tomography
Authors:
Hyoung Suk Park,
Chang Min Hyun,
Jin Keun Seo
Abstract:
This paper describes the mathematical structure of the ill-posed nonlinear inverse problem of low-dose dental cone-beam computed tomography (CBCT) and explains the advantages of a deep learning-based approach to the reconstruction of computed tomography images over conventional regularization methods. This paper explains the underlying reasons why dental CBCT is more ill-posed than standard comput…
▽ More
This paper describes the mathematical structure of the ill-posed nonlinear inverse problem of low-dose dental cone-beam computed tomography (CBCT) and explains the advantages of a deep learning-based approach to the reconstruction of computed tomography images over conventional regularization methods. This paper explains the underlying reasons why dental CBCT is more ill-posed than standard computed tomography. Despite this severe ill-posedness, the demand for dental CBCT systems is rapidly growing because of their cost competitiveness and low radiation dose. We then describe the limitations of existing methods in the accurate restoration of the morphological structures of teeth using dental CBCT data severely damaged by metal implants. We further discuss the usefulness of panoramic images generated from CBCT data for accurate tooth segmentation. We also discuss the possibility of utilizing radiation-free intra-oral scan data as prior information in CBCT image reconstruction to compensate for the damage to data caused by metal implants.
△ Less
Submitted 2 March, 2023;
originally announced March 2023.
-
Metal Artifact Reduction with Intra-Oral Scan Data for 3D Low Dose Maxillofacial CBCT Modeling
Authors:
Chang Min Hyun,
Taigyntuya Bayaraa,
Hye Sun Yun,
Tae Jun Jang,
Hyoung Suk Park,
Jin Keun Seo
Abstract:
Low-dose dental cone beam computed tomography (CBCT) has been increasingly used for maxillofacial modeling. However, the presence of metallic inserts, such as implants, crowns, and dental filling, causes severe streaking and shading artifacts in a CBCT image and loss of the morphological structures of the teeth, which consequently prevents accurate segmentation of bones. A two-stage metal artifact…
▽ More
Low-dose dental cone beam computed tomography (CBCT) has been increasingly used for maxillofacial modeling. However, the presence of metallic inserts, such as implants, crowns, and dental filling, causes severe streaking and shading artifacts in a CBCT image and loss of the morphological structures of the teeth, which consequently prevents accurate segmentation of bones. A two-stage metal artifact reduction method is proposed for accurate 3D low-dose maxillofacial CBCT modeling, where a key idea is to utilize explicit tooth shape prior information from intra-oral scan data whose acquisition does not require any extra radiation exposure. In the first stage, an image-to-image deep learning network is employed to mitigate metal-related artifacts. To improve the learning ability, the proposed network is designed to take advantage of the intra-oral scan data as side-inputs and perform multi-task learning of auxiliary tooth segmentation. In the second stage, a 3D maxillofacial model is constructed by segmenting the bones from the dental CBCT image corrected in the first stage. For accurate bone segmentation, weighted thresholding is applied, wherein the weighting region is determined depending on the geometry of the intra-oral scan data. Because acquiring a paired training dataset of metal-artifact-free and metal artifact-affected dental CBCT images is challenging in clinical practice, an automatic method of generating a realistic dataset according to the CBCT physics model is introduced. Numerical simulations and clinical experiments show the feasibility of the proposed method, which takes advantage of tooth surface information from intra-oral scan data in 3D low dose maxillofacial CBCT modeling.
△ Less
Submitted 7 February, 2022;
originally announced February 2022.
-
Fully automatic integration of dental CBCT images and full-arch intraoral impressions with stitching error correction via individual tooth segmentation and identification
Authors:
Tae Jun Jang,
Hye Sun Yun,
Chang Min Hyun,
Jong-Eun Kim,
Sang-Hwy Lee,
Jin Keun Seo
Abstract:
We present a fully automated method of integrating intraoral scan (IOS) and dental cone-beam computerized tomography (CBCT) images into one image by complementing each image's weaknesses. Dental CBCT alone may not be able to delineate precise details of the tooth surface due to limited image resolution and various CBCT artifacts, including metal-induced artifacts. IOS is very accurate for the scan…
▽ More
We present a fully automated method of integrating intraoral scan (IOS) and dental cone-beam computerized tomography (CBCT) images into one image by complementing each image's weaknesses. Dental CBCT alone may not be able to delineate precise details of the tooth surface due to limited image resolution and various CBCT artifacts, including metal-induced artifacts. IOS is very accurate for the scanning of narrow areas, but it produces cumulative stitching errors during full-arch scanning. The proposed method is intended not only to compensate the low-quality of CBCT-derived tooth surfaces with IOS, but also to correct the cumulative stitching errors of IOS across the entire dental arch. Moreover, the integration provide both gingival structure of IOS and tooth roots of CBCT in one image. The proposed fully automated method consists of four parts; (i) individual tooth segmentation and identification module for IOS data (TSIM-IOS); (ii) individual tooth segmentation and identification module for CBCT data (TSIM-CBCT); (iii) global-to-local tooth registration between IOS and CBCT; and (iv) stitching error correction of full-arch IOS. The experimental results show that the proposed method achieved landmark and surface distance errors of 112.4 $μ$m and 301.7 $μ$m, respectively.
△ Less
Submitted 2 March, 2023; v1 submitted 3 December, 2021;
originally announced December 2021.
-
Fast and Accurate Single-Image Depth Estimation on Mobile Devices, Mobile AI 2021 Challenge: Report
Authors:
Andrey Ignatov,
Grigory Malivenko,
David Plowman,
Samarth Shukla,
Radu Timofte,
Ziyu Zhang,
Yicheng Wang,
Zilong Huang,
Guozhong Luo,
Gang Yu,
Bin Fu,
Yiran Wang,
Xingyi Li,
Min Shi,
Ke Xian,
Zhiguo Cao,
Jin-Hua Du,
Pei-Lin Wu,
Chao Ge,
Jiaoyang Yao,
Fangwen Tu,
Bo Li,
Jung Eun Yoo,
Kwanggyoon Seo,
Jialei Xu
, et al. (13 additional authors not shown)
Abstract:
Depth estimation is an important computer vision problem with many practical applications to mobile devices. While many solutions have been proposed for this task, they are usually very computationally expensive and thus are not applicable for on-device inference. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based d…
▽ More
Depth estimation is an important computer vision problem with many practical applications to mobile devices. While many solutions have been proposed for this task, they are usually very computationally expensive and thus are not applicable for on-device inference. To address this problem, we introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based depth estimation solutions that can demonstrate a nearly real-time performance on smartphones and IoT platforms. For this, the participants were provided with a new large-scale dataset containing RGB-depth image pairs obtained with a dedicated stereo ZED camera producing high-resolution depth maps for objects located at up to 50 meters. The runtime of all models was evaluated on the popular Raspberry Pi 4 platform with a mobile ARM-based Broadcom chipset. The proposed solutions can generate VGA resolution depth maps at up to 10 FPS on the Raspberry Pi 4 while achieving high fidelity results, and are compatible with any Android or Linux-based mobile devices. A detailed description of all models developed in the challenge is provided in this paper.
△ Less
Submitted 17 May, 2021;
originally announced May 2021.
-
Deep Retinex Network for Estimating Illumination Colors with Self-Supervised Learning
Authors:
Kouki Seo,
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
We propose a novel Retinex image-decomposition network that can be trained in a self-supervised manner. The Retinex image-decomposition aims to decompose an image into illumination-invariant and illumination-variant components, referred to as "reflectance" and "shading," respectively. Although there are three consistencies that the reflectance and shading should satisfy, most conventional work con…
▽ More
We propose a novel Retinex image-decomposition network that can be trained in a self-supervised manner. The Retinex image-decomposition aims to decompose an image into illumination-invariant and illumination-variant components, referred to as "reflectance" and "shading," respectively. Although there are three consistencies that the reflectance and shading should satisfy, most conventional work considers only one or two of the consistencies. For this reason, the three consistencies are considered in the proposed network. In addition, by using generated pseudo-images for training, the proposed network can be trained with self-supervised learning. Experimental results show that our network can decompose images into reflectance and shading components. Furthermore, it is shown that the proposed network can be used for white-balance adjustment.
△ Less
Submitted 8 February, 2021;
originally announced February 2021.
-
Automated 3D cephalometric landmark identification using computerized tomography
Authors:
Hye Sun Yun,
Chang Min Hyun,
Seong Hyeon Baek,
Sang-Hwy Lee,
Jin Keun Seo
Abstract:
Identification of 3D cephalometric landmarks that serve as proxy to the shape of human skull is the fundamental step in cephalometric analysis. Since manual landmarking from 3D computed tomography (CT) images is a cumbersome task even for the trained experts, automatic 3D landmark detection system is in a great need. Recently, automatic landmarking of 2D cephalograms using deep learning (DL) has a…
▽ More
Identification of 3D cephalometric landmarks that serve as proxy to the shape of human skull is the fundamental step in cephalometric analysis. Since manual landmarking from 3D computed tomography (CT) images is a cumbersome task even for the trained experts, automatic 3D landmark detection system is in a great need. Recently, automatic landmarking of 2D cephalograms using deep learning (DL) has achieved great success, but 3D landmarking for more than 80 landmarks has not yet reached a satisfactory level, because of the factors hindering machine learning such as the high dimensionality of the input data and limited amount of training data due to ethical restrictions on the use of medical data. This paper presents a semi-supervised DL method for 3D landmarking that takes advantage of anonymized landmark dataset with paired CT data being removed. The proposed method first detects a small number of easy-to-find reference landmarks, then uses them to provide a rough estimation of the entire landmarks by utilizing the low dimensional representation learned by variational autoencoder (VAE). Anonymized landmark dataset is used for training the VAE. Finally, coarse-to-fine detection is applied to the small bounding box provided by rough estimation, using separate strategies suitable for mandible and cranium. For mandibular landmarks, patch-based 3D CNN is applied to the segmented image of the mandible (separated from the maxilla), in order to capture 3D morphological features of mandible associated with the landmarks. We detect 6 landmarks around the condyle all at once, instead of one by one, because they are closely related to each other. For cranial landmarks, we again use VAE-based latent representation for more accurate annotation. In our experiment, the proposed method achieved an averaged 3D point-to-point error of 2.91 mm for 90 landmarks only with 15 paired training data.
△ Less
Submitted 16 December, 2020;
originally announced January 2021.
-
Neural Crossbreed: Neural Based Image Metamorphosis
Authors:
Sanghun Park,
Kwanggyoon Seo,
Junyong Noh
Abstract:
We propose Neural Crossbreed, a feed-forward neural network that can learn a semantic change of input images in a latent space to create the morphing effect. Because the network learns a semantic change, a sequence of meaningful intermediate images can be generated without requiring the user to specify explicit correspondences. In addition, the semantic change learning makes it possible to perform…
▽ More
We propose Neural Crossbreed, a feed-forward neural network that can learn a semantic change of input images in a latent space to create the morphing effect. Because the network learns a semantic change, a sequence of meaningful intermediate images can be generated without requiring the user to specify explicit correspondences. In addition, the semantic change learning makes it possible to perform the morphing between the images that contain objects with significantly different poses or camera views. Furthermore, just as in conventional morphing techniques, our morphing network can handle shape and appearance transitions separately by disentangling the content and the style transfer for rich usability. We prepare a training dataset for morphing using a pre-trained BigGAN, which generates an intermediate image by interpolating two latent vectors at an intended morphing value. This is the first attempt to address image morphing using a pre-trained generative model in order to learn semantic transformation. The experiments show that Neural Crossbreed produces high quality morphed images, overcoming various limitations associated with conventional approaches. In addition, Neural Crossbreed can be further extended for diverse applications such as multi-image morphing, appearance transfer, and video frame interpolation.
△ Less
Submitted 2 September, 2020;
originally announced September 2020.
-
Hue-Correction Scheme Considering Non-Linear Camera Response for Multi-Exposure Image Fusion
Authors:
Kouki Seo,
Chihiro Go,
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
We propose a novel hue-correction scheme for multi-exposure image fusion (MEF). Various MEF methods have so far been studied to generate higher-quality images. However, there are few MEF methods considering hue distortion unlike other fields of image processing, due to a lack of a reference image that has correct hue. In the proposed scheme, we generate an HDR image as a reference for hue correcti…
▽ More
We propose a novel hue-correction scheme for multi-exposure image fusion (MEF). Various MEF methods have so far been studied to generate higher-quality images. However, there are few MEF methods considering hue distortion unlike other fields of image processing, due to a lack of a reference image that has correct hue. In the proposed scheme, we generate an HDR image as a reference for hue correction, from input multi-exposure images. After that, hue distortion in images fused by an MEF method is removed by using hue information of the HDR one, on the basis of the constant-hue plane in the RGB color space. In simulations, the proposed scheme is demonstrated to be effective to correct hue-distortion caused by conventional MEF methods. Experimental results also show that the proposed scheme can generate high-quality images, regardless of exposure conditions of input multi-exposure images.
△ Less
Submitted 18 July, 2020;
originally announced July 2020.
-
Hue Correction Scheme for Multi-Exposure Image Fusion Considering Hue Distortion in Input Images
Authors:
Kouki Seo,
Chihiro Go,
Yuma Kinoshita,
Hitoshi Kiya
Abstract:
We propose a novel hue-correction scheme for multi-exposure image fusion (MEF). Various MEF methods have so far been studied to generate higher-quality images. However, there are few MEF methods considering hue distortion unlike other fields of image processing, due to a lack of a reference image that has correct hue. In the proposed scheme, we generate an HDR image as a reference for hue correcti…
▽ More
We propose a novel hue-correction scheme for multi-exposure image fusion (MEF). Various MEF methods have so far been studied to generate higher-quality images. However, there are few MEF methods considering hue distortion unlike other fields of image processing, due to a lack of a reference image that has correct hue. In the proposed scheme, we generate an HDR image as a reference for hue correction, from input multi-exposure images. After that, hue distortion in an image fused by an MEF method is removed by using hue information of the HDR one, on the basis of the constant-hue plane in the RGB color space. In simulations, the proposed scheme is demonstrated to be effective to correct hue-distortion caused by conventional MEF methods. Experimental results also show that the proposed scheme can generate high-quality images, regardless of exposure conditions of input multi-exposure images.
△ Less
Submitted 5 May, 2020;
originally announced May 2020.
-
Prediction of Individual Propofol Requirements based on Preoperative EEG Signals
Authors:
Young-Seok Kweon,
Minji Lee,
Dong-Ok Won,
Kwang-Suk Seo
Abstract:
The patient must be given an adequate amount of propofol for safe surgery since overcapacity and low capacity cause accidents. However, the sensitivity of propofol varies from patient to patient, making it very difficult to determine the propofol requirements for anesthesia. This paper aims to propose a neurophysiological predictor of propofol requirements based on the preoperative electroencephal…
▽ More
The patient must be given an adequate amount of propofol for safe surgery since overcapacity and low capacity cause accidents. However, the sensitivity of propofol varies from patient to patient, making it very difficult to determine the propofol requirements for anesthesia. This paper aims to propose a neurophysiological predictor of propofol requirements based on the preoperative electroencephalogram (EEG). We exploited the canonical correlation analysis that infers the amount of information on the propofol requirements. The results showed that the preoperative EEG included the factor that could explain the propofol requirements. Specifically, the frontal and posterior regions had crucial information on the propofol requirements. Moreover, there was a significantly different power in the frontal and posterior regions between baseline and unconsciousness periods, unlike the alpha power in the central region. These findings showed the potential that preoperative EEG could predict the propofol requirements.
△ Less
Submitted 10 February, 2020;
originally announced February 2020.
-
Deep Learning-Based Solvability of Underdetermined Inverse Problems in Medical Imaging
Authors:
Chang Min Hyun,
Seong Hyeon Baek,
Mingyu Lee,
Sung Min Lee,
Jin Keun Seo
Abstract:
Recently, with the significant developments in deep learning techniques, solving underdetermined inverse problems has become one of the major concerns in the medical imaging domain. Typical examples include undersampled magnetic resonance imaging, interior tomography, and sparse-view computed tomography, where deep learning techniques have achieved excellent performances. Although deep learning me…
▽ More
Recently, with the significant developments in deep learning techniques, solving underdetermined inverse problems has become one of the major concerns in the medical imaging domain. Typical examples include undersampled magnetic resonance imaging, interior tomography, and sparse-view computed tomography, where deep learning techniques have achieved excellent performances. Although deep learning methods appear to overcome the limitations of existing mathematical methods when handling various underdetermined problems, there is a lack of rigorous mathematical foundations that would allow us to elucidate the reasons for the remarkable performance of deep learning methods. This study focuses on learning the causal relationship regarding the structure of the training data suitable for deep learning, to solve highly underdetermined inverse problems. We observe that a majority of the problems of solving underdetermined linear systems in medical imaging are highly non-linear. Furthermore, we analyze if a desired reconstruction map can be learnable from the training data and underdetermined system.
△ Less
Submitted 25 June, 2020; v1 submitted 6 January, 2020;
originally announced January 2020.
-
A hue-preserving tone mapping scheme based on constant-hue plane without gamut problem
Authors:
Yuma Kinoshita,
Kouki Seo,
Artit Visavakitcharoen,
Hitoshi Kiya
Abstract:
We propose a novel hue-preserving tone mapping scheme. Various tone mapping operations have been studied so far, but there are very few works on color distortion caused in image tone mapping. First, LDR images produced from HDR ones by using conventional tone mapping operators (TMOs) are pointed out to have some distortion in hue values due to clipping and rounding quantization processing. Next,we…
▽ More
We propose a novel hue-preserving tone mapping scheme. Various tone mapping operations have been studied so far, but there are very few works on color distortion caused in image tone mapping. First, LDR images produced from HDR ones by using conventional tone mapping operators (TMOs) are pointed out to have some distortion in hue values due to clipping and rounding quantization processing. Next,we propose a novel method which allows LDR images to have the same maximally saturated color values as those of HDR ones. Generated LDR images by the proposed method have smaller hue degradation than LDR ones generated by conventional TMOs. Moreover, the proposed method is applicable to any TMOs. In an experiment, the proposed method is demonstrated not only to produce images with small hue degradation but also to maintain well-mapped luminance, in terms of three objective metrics: TMQI, hue value in CIEDE2000, and the maximally saturated color on the constant-hue plane in the RGB color space.
△ Less
Submitted 18 August, 2019;
originally announced August 2019.
-
Framelet Pooling Aided Deep Learning Network : The Method to Process High Dimensional Medical Data
Authors:
Chang Min Hyun,
Kang Cheol Kim,
Hyun Cheol Cho,
Jae Kyu Choi,
Jin Keun Seo
Abstract:
Machine learning-based analysis of medical images often faces several hurdles, such as the lack of training data, the curse of dimensionality problem, and the generalization issues. One of the main difficulties is that there exists computational cost problem in dealing with input data of large size matrices which represent medical images. The purpose of this paper is to introduce a framelet-poolin…
▽ More
Machine learning-based analysis of medical images often faces several hurdles, such as the lack of training data, the curse of dimensionality problem, and the generalization issues. One of the main difficulties is that there exists computational cost problem in dealing with input data of large size matrices which represent medical images. The purpose of this paper is to introduce a framelet-pooling aided deep learning method for mitigating computational bundle, caused by large dimensionality. By transforming high dimensional data into low dimensional components by filter banks with preserving detailed information, the proposed method aims to reduce the complexity of the neural network and computational costs significantly during the learning process. Various experiments show that our method is comparable to the standard unreduced learning method, while reducing computational burdens by decomposing large-sized learning tasks into several small-scale learning tasks.
△ Less
Submitted 25 July, 2019;
originally announced July 2019.
