-
Bézier Flow: a Surface-wise Gradient Descent Method for Multi-objective Optimization
Authors:
Akiyoshi Sannai,
Yasunari Hikima,
Ken Kobayashi,
Akinori Tanaka,
Naoki Hamada
Abstract:
In this paper, we propose a strategy to construct a multi-objective optimization algorithm from a single-objective optimization algorithm by using the Bézier simplex model. Also, we extend the stability of optimization algorithms in the sense of Probability Approximately Correct (PAC) learning and define the PAC stability. We prove that it leads to an upper bound on the generalization with high pr…
▽ More
In this paper, we propose a strategy to construct a multi-objective optimization algorithm from a single-objective optimization algorithm by using the Bézier simplex model. Also, we extend the stability of optimization algorithms in the sense of Probability Approximately Correct (PAC) learning and define the PAC stability. We prove that it leads to an upper bound on the generalization with high probability. Furthermore, we show that multi-objective optimization algorithms derived from a gradient descent-based single-objective optimization algorithm are PAC stable. We conducted numerical experiments and demonstrated that our method achieved lower generalization errors than the existing multi-objective optimization algorithm.
△ Less
Submitted 23 May, 2022;
originally announced May 2022.
-
A Two-phase Framework with a Bézier Simplex-based Interpolation Method for Computationally Expensive Multi-objective Optimization
Authors:
Ryoji Tanabe,
Youhei Akimoto,
Ken Kobayashi,
Hiroshi Umeki,
Shinichi Shirakawa,
Naoki Hamada
Abstract:
This paper proposes a two-phase framework with a Bézier simplex-based interpolation method (TPB) for computationally expensive multi-objective optimization. The first phase in TPB aims to approximate a few Pareto optimal solutions by optimizing a sequence of single-objective scalar problems. The first phase in TPB can fully exploit a state-of-the-art single-objective derivative-free optimizer. The…
▽ More
This paper proposes a two-phase framework with a Bézier simplex-based interpolation method (TPB) for computationally expensive multi-objective optimization. The first phase in TPB aims to approximate a few Pareto optimal solutions by optimizing a sequence of single-objective scalar problems. The first phase in TPB can fully exploit a state-of-the-art single-objective derivative-free optimizer. The second phase in TPB utilizes a Bézier simplex model to interpolate the solutions obtained in the first phase. The second phase in TPB fully exploits the fact that a Bézier simplex model can approximate the Pareto optimal solution set by exploiting its simplex structure when a given problem is simplicial. We investigate the performance of TPB on the 55 bi-objective BBOB problems. The results show that TPB performs significantly better than HMO-CMA-ES and some state-of-the-art meta-model-based optimizers.
△ Less
Submitted 29 March, 2022;
originally announced March 2022.
-
GenéLive! Generating Rhythm Actions in Love Live!
Authors:
Atsushi Takada,
Daichi Yamazaki,
Likun Liu,
Yudai Yoshida,
Nyamkhuu Ganbat,
Takayuki Shimotomai,
Taiga Yamamoto,
Daisuke Sakurai,
Naoki Hamada
Abstract:
This article presents our generative model for rhythm action games together with applications in business operations. Rhythm action games are video games in which the player is challenged to issue commands at the right timings during a music session. The timings are rendered in the chart, which consists of visual symbols, called notes, flying through the screen. We introduce our deep generative mo…
▽ More
This article presents our generative model for rhythm action games together with applications in business operations. Rhythm action games are video games in which the player is challenged to issue commands at the right timings during a music session. The timings are rendered in the chart, which consists of visual symbols, called notes, flying through the screen. We introduce our deep generative model, GenéLive!, which outperforms the state-of-the-art model by taking into account musical structures through beats and temporal scales. Thanks to its favorable performance, GenéLive! was put into operation at KLab Inc., a Japan-based video game developer, and reduced the business cost of chart generation by as much as half. The application target included the phenomenal "Love Live!," which has more than 10 million users across Asia and beyond, and is one of the few rhythm action franchises that has led the online era of the genre. In this article, we evaluate the generative performance of GenéLive! using production datasets at KLab as well as open datasets for reproducibility, while the model continues to operate in their business. Our code and the model, tuned and trained using a supercomputer, are publicly available.
△ Less
Submitted 20 December, 2022; v1 submitted 25 February, 2022;
originally announced February 2022.
-
Explicitly Multi-Modal Benchmarks for Multi-Objective Optimization
Authors:
Ryosuke Ota,
Reiya Hagiwara,
Naoki Hamada,
Likun Liu,
Takahiro Yamamoto,
Daisuke Sakurai
Abstract:
In multi-objective optimization, designing good benchmark problems is an important issue for improving solvers.
Controlling the global location of Pareto optima in existing benchmark problems has been problematic, and it is even more difficult when the design space is high-dimensional since visualization is extremely challenging.
As a benchmarking with explicit local Pareto fronts, we introduc…
▽ More
In multi-objective optimization, designing good benchmark problems is an important issue for improving solvers.
Controlling the global location of Pareto optima in existing benchmark problems has been problematic, and it is even more difficult when the design space is high-dimensional since visualization is extremely challenging.
As a benchmarking with explicit local Pareto fronts, we introduce a benchmarking based on basin connectivity (3BC) by using basins of attraction.
The 3BC allows for the specification of a multimodal landscape through a kind of topological analysis called the basin graph, effectively generating optimization problems from this graph.
Various known indicators measure the performance of a solver in searching global Pareto optima, but using 3BC can make us localize them for each local Pareto front by restricting it to its basin.
3BC's mathematical formulation ensures the accurate representation of the specified optimization landscape, guaranteeing the existence of intended local and global Pareto optima.
△ Less
Submitted 9 February, 2024; v1 submitted 7 October, 2021;
originally announced October 2021.
-
All unconstrained strongly convex problems are weakly simplicial
Authors:
Yusuke Mizota,
Naoki Hamada,
Shunsuke Ichiki
Abstract:
A multi-objective optimization problem is $C^r$ weakly simplicial if there exists a $C^r$ surjection from a simplex onto the Pareto set/front such that the image of each subsimplex is the Pareto set/front of a subproblem, where $0\leq r\leq \infty$. This property is helpful to compute a parametric-surface approximation of the entire Pareto set and Pareto front. It is known that all unconstrained s…
▽ More
A multi-objective optimization problem is $C^r$ weakly simplicial if there exists a $C^r$ surjection from a simplex onto the Pareto set/front such that the image of each subsimplex is the Pareto set/front of a subproblem, where $0\leq r\leq \infty$. This property is helpful to compute a parametric-surface approximation of the entire Pareto set and Pareto front. It is known that all unconstrained strongly convex $C^r$ problems are $C^{r-1}$ weakly simplicial for $1\leq r \leq \infty$. In this paper, we show that all unconstrained strongly convex problems are $C^0$ weakly simplicial. The usefulness of this theorem is demonstrated in a sparse modeling application: we reformulate the elastic net as a non-differentiable multi-objective strongly convex problem and approximate its Pareto set (the set of all trained models with different hyper-parameters) and Pareto front (the set of performance metrics of the trained models) by using a Bézier simplex fitting method, which accelerates hyper-parameter search.
△ Less
Submitted 23 June, 2021;
originally announced June 2021.
-
Approximate Bayesian Computation of Bézier Simplices
Authors:
Akinori Tanaka,
Akiyoshi Sannai,
Ken Kobayashi,
Naoki Hamada
Abstract:
Bézier simplex fitting algorithms have been recently proposed to approximate the Pareto set/front of multi-objective continuous optimization problems. These new methods have shown to be successful at approximating various shapes of Pareto sets/fronts when sample points exactly lie on the Pareto set/front. However, if the sample points scatter away from the Pareto set/front, those methods often lik…
▽ More
Bézier simplex fitting algorithms have been recently proposed to approximate the Pareto set/front of multi-objective continuous optimization problems. These new methods have shown to be successful at approximating various shapes of Pareto sets/fronts when sample points exactly lie on the Pareto set/front. However, if the sample points scatter away from the Pareto set/front, those methods often likely suffer from over-fitting. To overcome this issue, in this paper, we extend the Bézier simplex model to a probabilistic one and propose a new learning algorithm of it, which falls into the framework of approximate Bayesian computation (ABC) based on the Wasserstein distance. We also study the convergence property of the Wasserstein ABC algorithm. An extensive experimental evaluation on publicly available problem instances shows that the new algorithm converges on a finite sample. Moreover, it outperforms the deterministic fitting methods on noisy instances.
△ Less
Submitted 12 April, 2021; v1 submitted 10 April, 2021;
originally announced April 2021.
-
Asymptotic Risk of Bezier Simplex Fitting
Authors:
Akinori Tanaka,
Akiyoshi Sannai,
Ken Kobayashi,
Naoki Hamada
Abstract:
The Bezier simplex fitting is a novel data modeling technique which exploits geometric structures of data to approximate the Pareto front of multi-objective optimization problems. There are two fitting methods based on different sampling strategies. The inductive skeleton fitting employs a stratified subsampling from each skeleton of a simplex, whereas the all-at-once fitting uses a non-stratified…
▽ More
The Bezier simplex fitting is a novel data modeling technique which exploits geometric structures of data to approximate the Pareto front of multi-objective optimization problems. There are two fitting methods based on different sampling strategies. The inductive skeleton fitting employs a stratified subsampling from each skeleton of a simplex, whereas the all-at-once fitting uses a non-stratified sampling which treats a simplex as a whole. In this paper, we analyze the asymptotic risks of those Bézier simplex fitting methods and derive the optimal subsample ratio for the inductive skeleton fitting. It is shown that the inductive skeleton fitting with the optimal ratio has a smaller risk when the degree of a Bezier simplex is less than three. Those results are verified numerically under small to moderate sample sizes. In addition, we provide two complementary applications of our theory: a generalized location problem and a multi-objective hyper-parameter tuning of the group lasso. The former can be represented by a Bezier simplex of degree two where the inductive skeleton fitting outperforms. The latter can be represented by a Bezier simplex of degree three where the all-at-once fitting gets an advantage.
△ Less
Submitted 17 June, 2019;
originally announced June 2019.
-
Data-Driven Analysis of Pareto Set Topology
Authors:
Naoki Hamada,
Keisuke Goto
Abstract:
When and why can evolutionary multi-objective optimization (EMO) algorithms cover the entire Pareto set? That is a major concern for EMO researchers and practitioners. A recent theoretical study revealed that (roughly speaking) if the Pareto set forms a topological simplex (a curved line, a curved triangle, a curved tetrahedron, etc.), then decomposition-based EMO algorithms can cover the entire P…
▽ More
When and why can evolutionary multi-objective optimization (EMO) algorithms cover the entire Pareto set? That is a major concern for EMO researchers and practitioners. A recent theoretical study revealed that (roughly speaking) if the Pareto set forms a topological simplex (a curved line, a curved triangle, a curved tetrahedron, etc.), then decomposition-based EMO algorithms can cover the entire Pareto set. Usually, we cannot know the true Pareto set and have to estimate its topology by using the population of EMO algorithms during or after the runtime. This paper presents a data-driven approach to analyze the topology of the Pareto set. We give a theory of how to recognize the topology of the Pareto set from data and implement an algorithm to judge whether the true Pareto set may form a topological simplex or not. Numerical experiments show that the proposed method correctly recognizes the topology of high-dimensional Pareto sets within reasonable population size.
△ Less
Submitted 19 April, 2018;
originally announced April 2018.
-
Simple Problems: The Simplicial Gluing Structure of Pareto Sets and Pareto Fronts
Authors:
Naoki Hamada
Abstract:
Quite a few studies on real-world applications of multi-objective optimization reported that their Pareto sets and Pareto fronts form a topological simplex. Such a class of problems was recently named the simple problems, and their Pareto set and Pareto front were observed to have a gluing structure similar to the faces of a simplex. This paper gives a theoretical justification for that observatio…
▽ More
Quite a few studies on real-world applications of multi-objective optimization reported that their Pareto sets and Pareto fronts form a topological simplex. Such a class of problems was recently named the simple problems, and their Pareto set and Pareto front were observed to have a gluing structure similar to the faces of a simplex. This paper gives a theoretical justification for that observation by proving the gluing structure of the Pareto sets/fronts of subproblems of a simple problem. The simplicity of standard benchmark problems is studied.
△ Less
Submitted 18 April, 2017;
originally announced September 2017.
-
Population Synthesis via k-Nearest Neighbor Crossover Kernel
Authors:
Naoki Hamada,
Katsumi Homma,
Hiroyuki Higuchi,
Hideyuki Kikuchi
Abstract:
The recent development of multi-agent simulations brings about a need for population synthesis. It is a task of reconstructing the entire population from a sampling survey of limited size (1% or so), supplying the initial conditions from which simulations begin. This paper presents a new kernel density estimator for this task. Our method is an analogue of the classical Breiman-Meisel-Purcell estim…
▽ More
The recent development of multi-agent simulations brings about a need for population synthesis. It is a task of reconstructing the entire population from a sampling survey of limited size (1% or so), supplying the initial conditions from which simulations begin. This paper presents a new kernel density estimator for this task. Our method is an analogue of the classical Breiman-Meisel-Purcell estimator, but employs novel techniques that harness the huge degree of freedom which is required to model high-dimensional nonlinearly correlated datasets: the crossover kernel, the k-nearest neighbor restriction of the kernel construction set and the bagging of kernels. The performance as a statistical estimator is examined through real and synthetic datasets. We provide an "optimization-free" parameter selection rule for our method, a theory of how our method works and a computational cost analysis. To demonstrate the usefulness as a population synthesizer, our method is applied to a household synthesis task for an urban micro-simulator.
△ Less
Submitted 26 August, 2015;
originally announced August 2015.
