-
Convergence of Sharpness-Aware Minimization Algorithms using Increasing Batch Size and Decaying Learning Rate
Authors:
Hinata Harada,
Hideaki Iiduka
Abstract:
The sharpness-aware minimization (SAM) algorithm and its variants, including gap guided SAM (GSAM), have been successful at improving the generalization capability of deep neural network models by finding flat local minima of the empirical loss in training. Meanwhile, it has been shown theoretically and practically that increasing the batch size or decaying the learning rate avoids sharp local min…
▽ More
The sharpness-aware minimization (SAM) algorithm and its variants, including gap guided SAM (GSAM), have been successful at improving the generalization capability of deep neural network models by finding flat local minima of the empirical loss in training. Meanwhile, it has been shown theoretically and practically that increasing the batch size or decaying the learning rate avoids sharp local minima of the empirical loss. In this paper, we consider the GSAM algorithm with increasing batch sizes or decaying learning rates, such as cosine annealing or linear learning rate, and theoretically show its convergence. Moreover, we numerically compare SAM (GSAM) with and without an increasing batch size and conclude that using an increasing batch size or decaying learning rate finds flatter local minima than using a constant batch size and learning rate.
△ Less
Submitted 16 September, 2024;
originally announced September 2024.
-
Generalization of pixel-wise phase estimation by CNN and improvement of phase-unwrapping by MRF optimization for one-shot 3D scan
Authors:
Hiroto Harada,
Michihiro Mikamo,
Ryo Furukawa,
Ryushuke Sagawa,
Hiroshi Kawasaki
Abstract:
Active stereo technique using single pattern projection, a.k.a. one-shot 3D scan, have drawn a wide attention from industry, medical purposes, etc. One severe drawback of one-shot 3D scan is sparse reconstruction. In addition, since spatial pattern becomes complicated for the purpose of efficient embedding, it is easily affected by noise, which results in unstable decoding. To solve the problems,…
▽ More
Active stereo technique using single pattern projection, a.k.a. one-shot 3D scan, have drawn a wide attention from industry, medical purposes, etc. One severe drawback of one-shot 3D scan is sparse reconstruction. In addition, since spatial pattern becomes complicated for the purpose of efficient embedding, it is easily affected by noise, which results in unstable decoding. To solve the problems, we propose a pixel-wise interpolation technique for one-shot scan, which is applicable to any types of static pattern if the pattern is regular and periodic. This is achieved by U-net which is pre-trained by CG with efficient data augmentation algorithm. In the paper, to further overcome the decoding instability, we propose a robust correspondence finding algorithm based on Markov random field (MRF) optimization. We also propose a shape refinement algorithm based on b-spline and Gaussian kernel interpolation using explicitly detected laser curves. Experiments are conducted to show the effectiveness of the proposed method using real data with strong noises and textures.
△ Less
Submitted 26 September, 2023;
originally announced September 2023.
-
Licensed-Assisted Access to Unlicensed Spectrum in LTE Release 13
Authors:
Hwan-Joon,
Kwon,
Jeongho Jeon,
Abhijeet Bhorkar,
Qiaoyang Ye,
Hiroki Harada,
Yu Jiang,
Liu Liu,
Satoshi Nagata,
Boon Loong Ng,
Thomas Novlan,
Jinyoung Oh,
Wang Yi
Abstract:
Exploiting the unlicensed spectrum is considered by 3GPP as one promising solution to meet the ever-increasing traffic growth. As a result, one major enhancement for LTE in Release 13 has been to enable its operation in the unlicensed spectrum via Licensed-Assisted Access (LAA). In this article, we provide an overview of the Release 13 LAA technology including motivation, use cases, LTE enhancemen…
▽ More
Exploiting the unlicensed spectrum is considered by 3GPP as one promising solution to meet the ever-increasing traffic growth. As a result, one major enhancement for LTE in Release 13 has been to enable its operation in the unlicensed spectrum via Licensed-Assisted Access (LAA). In this article, we provide an overview of the Release 13 LAA technology including motivation, use cases, LTE enhancements for enabling the unlicensed band operation, and the coexistence evaluation results contributed by 3GPP participants.
△ Less
Submitted 28 April, 2016;
originally announced April 2016.
-
Practical Design for Multiple-Antenna Cognitive Radio Networks with Coexistence Constraint
Authors:
Pin-Hsun Lin,
Gabriel P. Villardi,
Zhou Lan,
Hiroshi Harada
Abstract:
In this paper we investigate the practical design for the multiple-antenna cognitive radio (CR) networks sharing the geographically used or unused spectrum. We consider a single cell network formed by the primary users (PU), which are half-duplex two-hop relay channels and the secondary users (SU) are single user additive white Gaussian noise channels. In addition, the coexistence constraint which…
▽ More
In this paper we investigate the practical design for the multiple-antenna cognitive radio (CR) networks sharing the geographically used or unused spectrum. We consider a single cell network formed by the primary users (PU), which are half-duplex two-hop relay channels and the secondary users (SU) are single user additive white Gaussian noise channels. In addition, the coexistence constraint which requires PUs' coding schemes and rates unchanged with the emergence of SU, should be satisfied. The contribution of this paper are twofold. First, we explicitly design the scheme to pair the SUs to the existing PUs in a single cell network. Second, we jointly design the nonlinear precoder, relay beamformer, and the transmitter and receiver beamformers to minimize the sum mean square error of the SU system. In the first part, we derive an approximate relation between the relay ratio, chordal distance and strengths of the vector channels, and the transmit powers. Based on this relation, we are able to solve the optimal pairing between SUs and PUs efficiently. In the second part, considering the feasibility of implementation, we exploit the Tomlinson-Harashima precoding instead of the dirty paper coding to mitigate the interference at the SU receiver, which is known side information at the SU transmitter. To complete the design, we first approximate the optimization problem as a convex one. Then we propose an iterative algorithm to solve it with CVX. This joint design exploits all the degrees of design. To the best of our knowledge, both the two parts have never been considered in the literature. Numerical results show that the proposed pairing scheme outperforms the greedy and random pairing with low complexity. Numerical results also show that even if all the channel matrices are full rank, under which the simple zero forcing scheme is infeasible, the proposed scheme can still work well.
△ Less
Submitted 9 December, 2013;
originally announced December 2013.
