-
Mean field variational framework for integer optimization
Authors:
Arturo Berrones,
Jonás Velasco,
Juan Banda
Abstract:
A principled method to obtain approximate solutions of general constrained integer optimization problems is introduced. The approach is based on the calculation of a mean field probability distribution for the decision variables which is consistent with the objective function and the constraints. The original discrete task is in this way transformed into a continuous variational problem. In the co…
▽ More
A principled method to obtain approximate solutions of general constrained integer optimization problems is introduced. The approach is based on the calculation of a mean field probability distribution for the decision variables which is consistent with the objective function and the constraints. The original discrete task is in this way transformed into a continuous variational problem. In the context of particular problem classes at small and medium sizes, the mean field results are comparable to those of standard specialized methods, while at large sized instances is capable to find feasible solutions in computation times for which standard approaches can't find any valuable result. The mean field variational framework remains valid for widely diverse problem structures so it represents a promising paradigm for large dimensional nonlinear combinatorial optimization tasks.
△ Less
Submitted 5 May, 2013;
originally announced May 2013.
-
An estimation of distribution algorithm with adaptive Gibbs sampling for unconstrained global optimization
Authors:
Jonás Velasco,
Mario A. Saucedo-Espinosa,
Hugo Jair Escalante,
Karlo Mendoza,
César Emilio Villarreal-Rodríguez,
Óscar L. Chacón-Mondragón,
Adrián Rodríguez,
Arturo Berrones
Abstract:
In this paper is proposed a new heuristic approach belonging to the field of evolutionary Estimation of Distribution Algorithms (EDAs). EDAs builds a probability model and a set of solutions is sampled from the model which characterizes the distribution of such solutions. The main framework of the proposed method is an estimation of distribution algorithm, in which an adaptive Gibbs sampling is us…
▽ More
In this paper is proposed a new heuristic approach belonging to the field of evolutionary Estimation of Distribution Algorithms (EDAs). EDAs builds a probability model and a set of solutions is sampled from the model which characterizes the distribution of such solutions. The main framework of the proposed method is an estimation of distribution algorithm, in which an adaptive Gibbs sampling is used to generate new promising solutions and, in combination with a local search strategy, it improves the individual solutions produced in each iteration. The Estimation of Distribution Algorithm with Adaptive Gibbs Sampling we are proposing in this paper is called AGEDA. We experimentally evaluate and compare this algorithm against two deterministic procedures and several stochastic methods in three well known test problems for unconstrained global optimization. It is empirically shown that our heuristic is robust in problems that involve three central aspects that mainly determine the difficulty of global optimization problems, namely high-dimensionality, multi-modality and non-smoothness.
△ Less
Submitted 29 May, 2013; v1 submitted 11 July, 2011;
originally announced July 2011.
-
Bayesian Inference Based on Stationary Fokker-Planck Sampling
Authors:
Arturo Berrones
Abstract:
A novel formalism for Bayesian learning in the context of complex inference models is proposed. The method is based on the use of the Stationary Fokker--Planck (SFP) approach to sample from the posterior density. Stationary Fokker--Planck sampling generalizes the Gibbs sampler algorithm for arbitrary and unknown conditional densities. By the SFP procedure approximate analytical expressions for t…
▽ More
A novel formalism for Bayesian learning in the context of complex inference models is proposed. The method is based on the use of the Stationary Fokker--Planck (SFP) approach to sample from the posterior density. Stationary Fokker--Planck sampling generalizes the Gibbs sampler algorithm for arbitrary and unknown conditional densities. By the SFP procedure approximate analytical expressions for the conditionals and marginals of the posterior can be constructed. At each stage of SFP, the approximate conditionals are used to define a Gibbs sampling process, which is convergent to the full joint posterior. By the analytical marginals efficient learning methods in the context of Artificial Neural Networks are outlined. Off--line and incremental Bayesian inference and Maximum Likelihood Estimation from the posterior is performed in classification and regression examples. A comparison of SFP with other Monte Carlo strategies in the general problem of sampling from arbitrary densities is also presented. It is shown that SFP is able to jump large low--probabilty regions without the need of a careful tuning of any step size parameter. In fact, the SFP method requires only a small set of meaningful parameters which can be selected following clear, problem--independent guidelines. The computation cost of SFP, measured in terms of loss function evaluations, grows linearly with the given model's dimension.
△ Less
Submitted 25 October, 2009; v1 submitted 8 January, 2009;
originally announced January 2009.
-
Characterization of the convergence of stationary Fokker-Planck learning
Authors:
Arturo Berrones
Abstract:
The convergence properties of the stationary Fokker-Planck algorithm for the estimation of the asymptotic density of stochastic search processes is studied. Theoretical and empirical arguments for the characterization of convergence of the estimation in the case of separable and nonseparable nonlinear optimization problems are given. Some implications of the convergence of stationary Fokker-Plan…
▽ More
The convergence properties of the stationary Fokker-Planck algorithm for the estimation of the asymptotic density of stochastic search processes is studied. Theoretical and empirical arguments for the characterization of convergence of the estimation in the case of separable and nonseparable nonlinear optimization problems are given. Some implications of the convergence of stationary Fokker-Planck learning for the inference of parameters in artificial neural network models are outlined.
△ Less
Submitted 2 July, 2009; v1 submitted 21 February, 2008;
originally announced February 2008.
-
Stationary probability density of stochastic search processes in global optimization
Authors:
Arturo Berrones
Abstract:
A method for the construction of approximate analytical expressions for the stationary marginal densities of general stochastic search processes is proposed. By the marginal densities, regions of the search space that with high probability contain the global optima can be readily defined. The density estimation procedure involves a controlled number of linear operations, with a computational cos…
▽ More
A method for the construction of approximate analytical expressions for the stationary marginal densities of general stochastic search processes is proposed. By the marginal densities, regions of the search space that with high probability contain the global optima can be readily defined. The density estimation procedure involves a controlled number of linear operations, with a computational cost per iteration that grows linearly with problem size.
△ Less
Submitted 18 October, 2007;
originally announced October 2007.
-
Knowledge Network Approach to Noise Reduction
Authors:
Arturo Berrones
Abstract:
Previous preliminary results on the application of knowledge networks to noise reduction in stationary harmonic and weakly chaotic signals are extended to more general cases. The formalism gives a novel algorithm from which statistical tests for the identification of deterministic behavior in noisy stationary time series can be constructed.
Previous preliminary results on the application of knowledge networks to noise reduction in stationary harmonic and weakly chaotic signals are extended to more general cases. The formalism gives a novel algorithm from which statistical tests for the identification of deterministic behavior in noisy stationary time series can be constructed.
△ Less
Submitted 6 September, 2006;
originally announced September 2006.
-
Persistence in a Simple Model for the Earth's Atmosphere Temperature Fluctuations
Authors:
Arturo Berrones
Abstract:
The effect caused by the presence of a number of distinct time scales in a simple stochastic model for the Earth's atmosphere temperature fluctuations is studied. The model is described by a dissipative dynamics consisting of a set of coupled stochastic evolution equations. The system shows features that resemble recent observations. In contrast to other approaches, like autoregressive models, t…
▽ More
The effect caused by the presence of a number of distinct time scales in a simple stochastic model for the Earth's atmosphere temperature fluctuations is studied. The model is described by a dissipative dynamics consisting of a set of coupled stochastic evolution equations. The system shows features that resemble recent observations. In contrast to other approaches, like autoregressive models, the fluctuations of the atmosphere's temperature depend on parameters with clear physical meaning. A reduced version of the model is constructed and its temporal autocovariance function is explicitly written.
△ Less
Submitted 30 August, 2005; v1 submitted 13 June, 2005;
originally announced June 2005.
-
De Gustibus Disputandum
Authors:
Franco Bagnoli,
Arturo Berrones,
Fabio Franci
Abstract:
We propose a simple method to predict individuals' expectations about products using a knowledge network. As a complementary result, we show that the method is able, under certain conditions, to extract hidden information at neural level from a customers' choices database.
We propose a simple method to predict individuals' expectations about products using a knowledge network. As a complementary result, we show that the method is able, under certain conditions, to extract hidden information at neural level from a customers' choices database.
△ Less
Submitted 17 June, 2003;
originally announced June 2003.
