Title:Solving Inverse Problems with REINFORCE

Abstract:In this paper, with rigorous derivations we formally introduce the usage of reinforcement learning to the field of inverse problems by designing an iterative algorithm, called REINFORCE-IP, for solving a general type of non-linear inverse problems. By setting specific probability densities of the action rule, we connect our approach to the conventional regularization methods of Tikhonov regularization and iterative regularization. For the numerical implementation of our approach, we parameterize the solution-searching rule with the help of neural networks and iteratively improve the parameter using a reinforcement learning algorithm -- REINFORCE. Under standard assumptions we prove the almost sure convergence of the parameter to a locally optimal value. Our work provides two typical examples (nonlinear integral equations and parameter identification problems in partial differential equations) on how reinforcement learning can be applied in solving non-linear inverse problems. Our numerical experiments show that REINFORCE-IP is an efficient algorithm that can escape from local minimums and identify multi-solutions for inverse problems with non-uniqueness.