-
UNet--: Memory-Efficient and Feature-Enhanced Network Architecture based on U-Net with Reduced Skip-Connections
Authors:
Lingxiao Yin,
Wei Tao,
Dongyue Zhao,
Tadayuki Ito,
Kinya Osa,
Masami Kato,
Tse-Wei Chen
Abstract:
U-Net models with encoder, decoder, and skip-connections components have demonstrated effectiveness in a variety of vision tasks. The skip-connections transmit fine-grained information from the encoder to the decoder. It is necessary to maintain the feature maps used by the skip-connections in memory before the decoding stage. Therefore, they are not friendly to devices with limited resource. In t…
▽ More
U-Net models with encoder, decoder, and skip-connections components have demonstrated effectiveness in a variety of vision tasks. The skip-connections transmit fine-grained information from the encoder to the decoder. It is necessary to maintain the feature maps used by the skip-connections in memory before the decoding stage. Therefore, they are not friendly to devices with limited resource. In this paper, we propose a universal method and architecture to reduce the memory consumption and meanwhile generate enhanced feature maps to improve network performance. To this end, we design a simple but effective Multi-Scale Information Aggregation Module (MSIAM) in the encoder and an Information Enhancement Module (IEM) in the decoder. The MSIAM aggregates multi-scale feature maps into single-scale with less memory. After that, the aggregated feature maps can be expanded and enhanced to multi-scale feature maps by the IEM. By applying the proposed method on NAFNet, a SOTA model in the field of image restoration, we design a memory-efficient and feature-enhanced network architecture, UNet--. The memory demand by the skip-connections in the UNet-- is reduced by 93.3%, while the performance is improved compared to NAFNet. Furthermore, we show that our proposed method can be generalized to multiple visual tasks, with consistent improvements in both memory consumption and network accuracy compared to the existing efficient architectures.
△ Less
Submitted 24 December, 2024;
originally announced December 2024.
-
Radar-based Measurement of the Body Movements of Multiple Students in Classroom Environments
Authors:
Yu Oshima,
Tianyi Wang,
Masaya Kato,
Haruto Kobayashi,
Itsuki Iwata,
Yuji Tanaka,
Shuqiong Wu,
Manabu Wakuta,
Masako Myowa,
Tomoko Nishimura,
Atsushi Senju,
Takuya Sakamoto
Abstract:
We demonstrate the feasibility of the radar-based measurement of body movements in scenarios involving multiple students using a pair of 79-GHz millimeter-wave radar systems with array antennas. We quantify the body motion using the Doppler frequency calculated from radar echoes. The measurement accuracy is evaluated for two experimental scenarios, namely university students in an office and eleme…
▽ More
We demonstrate the feasibility of the radar-based measurement of body movements in scenarios involving multiple students using a pair of 79-GHz millimeter-wave radar systems with array antennas. We quantify the body motion using the Doppler frequency calculated from radar echoes. The measurement accuracy is evaluated for two experimental scenarios, namely university students in an office and elementary school students in a classroom. The body movements measured using the two radar systems are compared to evaluate the repeatability and angle dependency of the measurement. Moreover, in the first scenario, we compare the radar-estimated body movement with subjective evaluation scores provided by two evaluators. In the first scenario, the coefficient of correlation between the radar-estimated body movement and the subjective evaluation score is 0.73 on average, with a maximum value of 0.97; in the second scenario, the average correlation coefficient of body movements measured using two radar systems is as high as 0.78. These results indicate that the proposed approach can be used to monitor the body movements of multiple students in realistic scenarios.
△ Less
Submitted 3 December, 2024;
originally announced December 2024.
-
Hardware Architecture of Embedded Inference Accelerator and Analysis of Algorithms for Depthwise and Large-Kernel Convolutions
Authors:
Tse-Wei Chen,
Wei Tao,
Deyu Wang,
Dongchao Wen,
Kinya Osa,
Masami Kato
Abstract:
In order to handle modern convolutional neural networks (CNNs) efficiently, a hardware architecture of CNN inference accelerator is proposed to handle depthwise convolutions and regular convolutions, which are both essential building blocks for embedded-computer-vision algorithms. Different from related works, the proposed architecture can support filter kernels with different sizes with high flex…
▽ More
In order to handle modern convolutional neural networks (CNNs) efficiently, a hardware architecture of CNN inference accelerator is proposed to handle depthwise convolutions and regular convolutions, which are both essential building blocks for embedded-computer-vision algorithms. Different from related works, the proposed architecture can support filter kernels with different sizes with high flexibility since it does not require extra costs for intra-kernel parallelism, and it can generate convolution results faster than the architecture of the related works. The experimental results show the importance of supporting depthwise convolutions and dilated convolutions with the proposed hardware architecture. In addition to depthwise convolutions with large-kernels, a new structure called DDC layer, which includes the combination of depthwise convolutions and dilated convolutions, is also analyzed in this paper. For face detection, the computational costs decrease by 30%, and the model size decreases by 20% when the DDC layers are applied to the network. For image classification, the accuracy is increased by 1% by simply replacing $3 \times 3$ filters with $5 \times 5$ filters in depthwise convolutions.
△ Less
Submitted 29 April, 2021;
originally announced April 2021.
