-
LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes
Authors:
Zefan Qu,
Ke Xu,
Gerhard Petrus Hancke,
Rynson W. H. Lau
Abstract:
Neural Radiance Fields (NeRFs) have shown remarkable performances in producing novel-view images from high-quality scene images. However, hand-held low-light photography challenges NeRFs as the captured images may simultaneously suffer from low visibility, noise, and camera shakes. While existing NeRF methods may handle either low light or motion, directly combining them or incorporating additiona…
▽ More
Neural Radiance Fields (NeRFs) have shown remarkable performances in producing novel-view images from high-quality scene images. However, hand-held low-light photography challenges NeRFs as the captured images may simultaneously suffer from low visibility, noise, and camera shakes. While existing NeRF methods may handle either low light or motion, directly combining them or incorporating additional image-based enhancement methods does not work as these degradation factors are highly coupled. We observe that noise in low-light images is always sharp regardless of camera shakes, which implies an implicit order of these degradation factors within the image formation process. To this end, we propose in this paper a novel model, named LuSh-NeRF, which can reconstruct a clean and sharp NeRF from a group of hand-held low-light images. The key idea of LuSh-NeRF is to sequentially model noise and blur in the images via multi-view feature consistency and frequency information of NeRF, respectively. Specifically, LuSh-NeRF includes a novel Scene-Noise Decomposition (SND) module for decoupling the noise from the scene representation and a novel Camera Trajectory Prediction (CTP) module for the estimation of camera motions based on low-frequency scene information. To facilitate training and evaluations, we construct a new dataset containing both synthetic and real images. Experiments show that LuSh-NeRF outperforms existing approaches. Our code and dataset can be found here: https://github.com/quzefan/LuSh-NeRF.
△ Less
Submitted 11 November, 2024;
originally announced November 2024.
-
Color Shift Estimation-and-Correction for Image Enhancement
Authors:
Yiyu Li,
Ke Xu,
Gerhard Petrus Hancke,
Rynson W. H. Lau
Abstract:
Images captured under sub-optimal illumination conditions may contain both over- and under-exposures. Current approaches mainly focus on adjusting image brightness, which may exacerbate the color tone distortion in under-exposed areas and fail to restore accurate colors in over-exposed regions. We observe that over- and under-exposed regions display opposite color tone distribution shifts with res…
▽ More
Images captured under sub-optimal illumination conditions may contain both over- and under-exposures. Current approaches mainly focus on adjusting image brightness, which may exacerbate the color tone distortion in under-exposed areas and fail to restore accurate colors in over-exposed regions. We observe that over- and under-exposed regions display opposite color tone distribution shifts with respect to each other, which may not be easily normalized in joint modeling as they usually do not have ``normal-exposed'' regions/pixels as reference. In this paper, we propose a novel method to enhance images with both over- and under-exposures by learning to estimate and correct such color shifts. Specifically, we first derive the color feature maps of the brightened and darkened versions of the input image via a UNet-based network, followed by a pseudo-normal feature generator to produce pseudo-normal color feature maps. We then propose a novel COlor Shift Estimation (COSE) module to estimate the color shifts between the derived brightened (or darkened) color feature maps and the pseudo-normal color feature maps. The COSE module corrects the estimated color shifts of the over- and under-exposed regions separately. We further propose a novel COlor MOdulation (COMO) module to modulate the separately corrected colors in the over- and under-exposed regions to produce the enhanced image. Comprehensive experiments show that our method outperforms existing approaches. Project webpage: https://github.com/yiyulics/CSEC.
△ Less
Submitted 29 May, 2024; v1 submitted 27 May, 2024;
originally announced May 2024.
-
Autonomous Interference Mapping for Industrial IoT Networks over Unlicensed Bands
Authors:
Simone Grimaldi,
Aamir Mahmood,
Syed Ali Hassan,
Gerhard Petrus Hancke,
Mikael Gidlund
Abstract:
The limited coexistence capabilities of current Internet-of-things (IoT) wireless standards produce inefficient spectrum utilization and mutual performance impairment. The entity of the issue escalates in industrial IoT (IIoT) applications, which instead have stringent quality-of-service requirements and exhibit very-low error tolerance. The constant growth of wireless applications over unlicensed…
▽ More
The limited coexistence capabilities of current Internet-of-things (IoT) wireless standards produce inefficient spectrum utilization and mutual performance impairment. The entity of the issue escalates in industrial IoT (IIoT) applications, which instead have stringent quality-of-service requirements and exhibit very-low error tolerance. The constant growth of wireless applications over unlicensed bands mandates then the adoption of dynamic spectrum access techniques, which can greatly benefit from interference mapping over multiple dimensions of the radio space. In this article, the authors analyze the critical role of real-time interference detection and classification mechanisms that rely on IIoT devices only, without the added complexity of specialized hardware. The trade-offs between classification performance and feasibility are analyzed in connection with the implementation on low-complexity IIoT devices. Moreover, the authors explain how to use such mechanisms for enabling IIoT networks to construct and maintain multidimensional interference maps at run-time in an autonomous fashion. Lastly, the authors give an overview of the opportunities and challenges of using interference maps to enhance the performance of IIoT networks under interference.
△ Less
Submitted 24 June, 2020;
originally announced June 2020.
-
Deformable Object Tracking with Gated Fusion
Authors:
Wenxi Liu,
Yibing Song,
Dengsheng Chen,
Shengfeng He,
Yuanlong Yu,
Tao Yan,
Gerhard P. Hancke,
Rynson W. H. Lau
Abstract:
The tracking-by-detection framework receives growing attentions through the integration with the Convolutional Neural Networks (CNNs). Existing tracking-by-detection based methods, however, fail to track objects with severe appearance variations. This is because the traditional convolutional operation is performed on fixed grids, and thus may not be able to find the correct response while the obje…
▽ More
The tracking-by-detection framework receives growing attentions through the integration with the Convolutional Neural Networks (CNNs). Existing tracking-by-detection based methods, however, fail to track objects with severe appearance variations. This is because the traditional convolutional operation is performed on fixed grids, and thus may not be able to find the correct response while the object is changing pose or under varying environmental conditions. In this paper, we propose a deformable convolution layer to enrich the target appearance representations in the tracking-by-detection framework. We aim to capture the target appearance variations via deformable convolution, which adaptively enhances its original features. In addition, we also propose a gated fusion scheme to control how the variations captured by the deformable convolution affect the original appearance. The enriched feature representation through deformable convolution facilitates the discrimination of the CNN classifier on the target object and background. Extensive experiments on the standard benchmarks show that the proposed tracker performs favorably against state-of-the-art methods.
△ Less
Submitted 11 April, 2019; v1 submitted 27 September, 2018;
originally announced September 2018.
