Showing 1–11 of 11 results for author: Goldberg, D E

iBOA: The Incremental Bayesian Optimization Algorithm

Authors: Martin Pelikan, Kumara Sastry, David E. Goldberg

Abstract: This paper proposes the incremental Bayesian optimization algorithm (iBOA), which modifies standard BOA by removing the population of solutions and using incremental updates of the Bayesian network. iBOA is shown to be able to learn and exploit unrestricted Bayesian networks using incremental techniques for updating both the structure as well as the parameters of the probabilistic model. This re… ▽ More This paper proposes the incremental Bayesian optimization algorithm (iBOA), which modifies standard BOA by removing the population of solutions and using incremental updates of the Bayesian network. iBOA is shown to be able to learn and exploit unrestricted Bayesian networks using incremental techniques for updating both the structure as well as the parameters of the probabilistic model. This represents an important step toward the design of competent incremental estimation of distribution algorithms that can solve difficult nearly decomposable problems scalably and reliably. △ Less

Submitted 20 January, 2008; originally announced January 2008.

Comments: Also available at the MEDAL web site, http://medal.cs.umsl.edu/

Report number: MEDAL Report No. 2008002 ACM Class: I.2.6; I.2.8; G.1.6

Journal ref: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2008), ACM Press, 455-462

Decomposable Problems, Niching, and Scalability of Multiobjective Estimation of Distribution Algorithms

Authors: Kumara Sastry, Martin Pelikan, David E. Goldberg

Abstract: The paper analyzes the scalability of multiobjective estimation of distribution algorithms (MOEDAs) on a class of boundedly-difficult additively-separable multiobjective optimization problems. The paper illustrates that even if the linkage is correctly identified, massive multimodality of the search problems can easily overwhelm the nicher and lead to exponential scale-up. Facetwise models are s… ▽ More The paper analyzes the scalability of multiobjective estimation of distribution algorithms (MOEDAs) on a class of boundedly-difficult additively-separable multiobjective optimization problems. The paper illustrates that even if the linkage is correctly identified, massive multimodality of the search problems can easily overwhelm the nicher and lead to exponential scale-up. Facetwise models are subsequently used to propose a growth rate of the number of differing substructures between the two objectives to avoid the niching method from being overwhelmed and lead to polynomial scalability of MOEDAs. △ Less

Submitted 12 February, 2005; originally announced February 2005.

Comments: Submitted to Genetic and Evolutionary Computation Conference, GECCO-2005

Report number: IlliGAL Report No. 2005004

Multiobjective hBOA, Clustering, and Scalability

Authors: Martin Pelikan, Kumara Sastry, David E. Goldberg

Abstract: This paper describes a scalable algorithm for solving multiobjective decomposable problems by combining the hierarchical Bayesian optimization algorithm (hBOA) with the nondominated sorting genetic algorithm (NSGA-II) and clustering in the objective space. It is first argued that for good scalability, clustering or some other form of niching in the objective space is necessary and the size of ea… ▽ More This paper describes a scalable algorithm for solving multiobjective decomposable problems by combining the hierarchical Bayesian optimization algorithm (hBOA) with the nondominated sorting genetic algorithm (NSGA-II) and clustering in the objective space. It is first argued that for good scalability, clustering or some other form of niching in the objective space is necessary and the size of each niche should be approximately equal. Multiobjective hBOA (mohBOA) is then described that combines hBOA, NSGA-II and clustering in the objective space. The algorithm mohBOA differs from the multiobjective variants of BOA and hBOA proposed in the past by including clustering in the objective space and allocating an approximately equally sized portion of the population to each cluster. The algorithm mohBOA is shown to scale up well on a number of problems on which standard multiobjective evolutionary algorithms perform poorly. △ Less

Submitted 7 February, 2005; originally announced February 2005.

Comments: Also IlliGAL Report No. 2005005 (http://www-illigal.ge.uiuc.edu/). Submitted to GECCO-2005

Report number: IlliGAL Report No. 2005005 ACM Class: I.2.8; I.2.6; G.1.6; I.5.3

Sub-structural Niching in Estimation of Distribution Algorithms

Authors: K. Sastry, H. A. Abbass, D. E. Goldberg, D. D. Johnson

Abstract: We propose a sub-structural niching method that fully exploits the problem decomposition capability of linkage-learning methods such as the estimation of distribution algorithms and concentrate on maintaining diversity at the sub-structural level. The proposed method consists of three key components: (1) Problem decomposition and sub-structure identification, (2) sub-structure fitness estimation… ▽ More We propose a sub-structural niching method that fully exploits the problem decomposition capability of linkage-learning methods such as the estimation of distribution algorithms and concentrate on maintaining diversity at the sub-structural level. The proposed method consists of three key components: (1) Problem decomposition and sub-structure identification, (2) sub-structure fitness estimation, and (3) sub-structural niche preservation. The sub-structural niching method is compared to restricted tournament selection (RTS)--a niching method used in hierarchical Bayesian optimization algorithm--with special emphasis on sustained preservation of multiple global solutions of a class of boundedly-difficult, additively-separable multimodal problems. The results show that sub-structural niching successfully maintains multiple global optima over large number of generations and does so with significantly less population than RTS. Additionally, the market share of each of the niche is much closer to the expected level in sub-structural niching when compared to RTS. △ Less

Submitted 3 February, 2005; originally announced February 2005.

Report number: IlliGAL Report No. 2005002

Sub-Structural Niching in Non-Stationary Environments

Authors: K. Sastry, H. A. Abbass, D. E. Goldberg

Abstract: Niching enables a genetic algorithm (GA) to maintain diversity in a population. It is particularly useful when the problem has multiple optima where the aim is to find all or as many as possible of these optima. When the fitness landscape of a problem changes overtime, the problem is called non--stationary, dynamic or time--variant problem. In these problems, niching can maintain useful solution… ▽ More Niching enables a genetic algorithm (GA) to maintain diversity in a population. It is particularly useful when the problem has multiple optima where the aim is to find all or as many as possible of these optima. When the fitness landscape of a problem changes overtime, the problem is called non--stationary, dynamic or time--variant problem. In these problems, niching can maintain useful solutions to respond quickly, reliably and accurately to a change in the environment. In this paper, we present a niching method that works on the problem substructures rather than the whole solution, therefore it has less space complexity than previously known niching mechanisms. We show that the method is responding accurately when environmental changes occur. △ Less

Submitted 3 February, 2005; originally announced February 2005.

Comments: Final version published in 2005 Australian Artificial Intelligence Conference, pp. 873--885

Report number: IlliGAL Report No. 2004035

Oiling the Wheels of Change: The Role of Adaptive Automatic Problem Decomposition in Non--Stationary Environments

Authors: H. A. Abbass, K. Sastry, D. E. Goldberg

Abstract: Genetic algorithms (GAs) that solve hard problems quickly, reliably and accurately are called competent GAs. When the fitness landscape of a problem changes overtime, the problem is called non--stationary, dynamic or time--variant problem. This paper investigates the use of competent GAs for optimizing non--stationary optimization problems. More specifically, we use an information theoretic appr… ▽ More Genetic algorithms (GAs) that solve hard problems quickly, reliably and accurately are called competent GAs. When the fitness landscape of a problem changes overtime, the problem is called non--stationary, dynamic or time--variant problem. This paper investigates the use of competent GAs for optimizing non--stationary optimization problems. More specifically, we use an information theoretic approach based on the minimum description length principle to adaptively identify regularities and substructures that can be exploited to respond quickly to changes in the environment. We also develop a special type of problems with bounded difficulties to test non--stationary optimization problems. The results provide new insights into non-stationary optimization problems and show that a search algorithm which automatically identifies and exploits possible decompositions is more robust and responds quickly to changes than a simple genetic algorithm. △ Less

Submitted 3 February, 2005; originally announced February 2005.

Population Sizing for Genetic Programming Based Upon Decision Making

Authors: K. Sastry, U. -M. O'Reilly, D. E. Goldberg

Abstract: This paper derives a population sizing relationship for genetic programming (GP). Following the population-sizing derivation for genetic algorithms in Goldberg, Deb, and Clark (1992), it considers building block decision making as a key facet. The analysis yields a GP-unique relationship because it has to account for bloat and for the fact that GP solutions often use subsolution multiple times.… ▽ More This paper derives a population sizing relationship for genetic programming (GP). Following the population-sizing derivation for genetic algorithms in Goldberg, Deb, and Clark (1992), it considers building block decision making as a key facet. The analysis yields a GP-unique relationship because it has to account for bloat and for the fact that GP solutions often use subsolution multiple times. The population-sizing relationship depends upon tree size, solution complexity, problem difficulty and building block expression probability. The relationship is used to analyze and empirically investigate population sizing for three model GP problems named ORDER, ON-OFF and LOUD. These problems exhibit bloat to differing extents and differ in whether their solutions require the use of a building block multiple times. △ Less

Submitted 3 February, 2005; originally announced February 2005.

Comments: Final version published in O'Reilly, U.-M., et al. (2004). Genetic Programming Theory and Practice II. Boston, MA: Kluwer Academic Publishers. 49--66

Report number: IlliGAL Report No. 2004028

Efficiency Enhancement of Genetic Algorithms via Building-Block-Wise Fitness Estimation

Authors: Kumara Sastry, Martin Pelikan, David E. Goldberg

Abstract: This paper studies fitness inheritance as an efficiency enhancement technique for a class of competent genetic algorithms called estimation distribution algorithms. Probabilistic models of important sub-solutions are developed to estimate the fitness of a proportion of individuals in the population, thereby avoiding computationally expensive function evaluations. The effect of fitness inheritanc… ▽ More This paper studies fitness inheritance as an efficiency enhancement technique for a class of competent genetic algorithms called estimation distribution algorithms. Probabilistic models of important sub-solutions are developed to estimate the fitness of a proportion of individuals in the population, thereby avoiding computationally expensive function evaluations. The effect of fitness inheritance on the convergence time and population sizing are modeled and the speed-up obtained through inheritance is predicted. The results show that a fitness-inheritance mechanism which utilizes information on building-block fitnesses provides significant efficiency enhancement. For additively separable problems, fitness inheritance reduces the number of function evaluations to about half and yields a speed-up of about 1.75--2.25. △ Less

Submitted 18 May, 2004; originally announced May 2004.

Comments: IEEE International Conference on Evolutionary Computation (CEC-2004)

Report number: IlliGAL Report No. 2004010 ACM Class: G.1.6; G.3; I.2.6; I.2.8

Designing Competent Mutation Operators via Probabilistic Model Building of Neighborhoods

Authors: Kumara Sastry, David E. Goldberg

Abstract: This paper presents a competent selectomutative genetic algorithm (GA), that adapts linkage and solves hard problems quickly, reliably, and accurately. A probabilistic model building process is used to automatically identify key building blocks (BBs) of the search problem. The mutation operator uses the probabilistic model of linkage groups to find the best among competing building blocks. The c… ▽ More This paper presents a competent selectomutative genetic algorithm (GA), that adapts linkage and solves hard problems quickly, reliably, and accurately. A probabilistic model building process is used to automatically identify key building blocks (BBs) of the search problem. The mutation operator uses the probabilistic model of linkage groups to find the best among competing building blocks. The competent selectomutative GA successfully solves additively separable problems of bounded difficulty, requiring only subquadratic number of function evaluations. The results show that for additively separable problems the probabilistic model building BB-wise mutation scales as O(2^km^{1.5}), and requires O(k^{0.5}logm) less function evaluations than its selectorecombinative counterpart, confirming theoretical results reported elsewhere (Sastry & Goldberg, 2004). △ Less

Submitted 18 May, 2004; originally announced May 2004.

Comments: Genetic and Evolutionary Computation Conference (GECCO-2004)

Report number: IlliGAL Report No. 2004006 ACM Class: G.1.6; G.3; I.2.6; I.2.8

Let's Get Ready to Rumble: Crossover Versus Mutation Head to Head

Authors: Kumara Sastry, David E. Goldberg

Abstract: This paper analyzes the relative advantages between crossover and mutation on a class of deterministic and stochastic additively separable problems. This study assumes that the recombination and mutation operators have the knowledge of the building blocks (BBs) and effectively exchange or search among competing BBs. Facetwise models of convergence time and population sizing have been used to det… ▽ More This paper analyzes the relative advantages between crossover and mutation on a class of deterministic and stochastic additively separable problems. This study assumes that the recombination and mutation operators have the knowledge of the building blocks (BBs) and effectively exchange or search among competing BBs. Facetwise models of convergence time and population sizing have been used to determine the scalability of each algorithm. The analysis shows that for additively separable deterministic problems, the BB-wise mutation is more efficient than crossover, while the crossover outperforms the mutation on additively separable problems perturbed with additive Gaussian noise. The results show that the speed-up of using BB-wise mutation on deterministic problems is O(k^{0.5}logm), where k is the BB size, and m is the number of BBs. Likewise, the speed-up of using crossover on stochastic problems with fixed noise variance is O(mk^{0.5}log m). △ Less

Submitted 18 May, 2004; originally announced May 2004.

Comments: Genetic and Evolutionary Computation Conference (GECCO-2004)

Report number: IlliGAL Report No. 2004005 ACM Class: G.1.6; G.3; I.2.6; I.2.8

Efficiency Enhancement of Probabilistic Model Building Genetic Algorithms

Authors: Kumara Sastry, David E. Goldberg, Martin Pelikan

Abstract: This paper presents two different efficiency-enhancement techniques for probabilistic model building genetic algorithms. The first technique proposes the use of a mutation operator which performs local search in the sub-solution neighborhood identified through the probabilistic model. The second technique proposes building and using an internal probabilistic model of the fitness along with the p… ▽ More This paper presents two different efficiency-enhancement techniques for probabilistic model building genetic algorithms. The first technique proposes the use of a mutation operator which performs local search in the sub-solution neighborhood identified through the probabilistic model. The second technique proposes building and using an internal probabilistic model of the fitness along with the probabilistic model of variable interactions. The fitness values of some offspring are estimated using the probabilistic model, thereby avoiding computationally expensive function evaluations. The scalability of the aforementioned techniques are analyzed using facetwise models for convergence time and population sizing. The speed-up obtained by each of the methods is predicted and verified with empirical results. The results show that for additively separable problems the competent mutation operator requires O(k 0.5 logm)--where k is the building-block size, and m is the number of building blocks--less function evaluations than its selectorecombinative counterpart. The results also show that the use of an internal probabilistic fitness model reduces the required number of function evaluations to as low as 1-10% and yields a speed-up of 2--50. △ Less

Submitted 18 May, 2004; originally announced May 2004.

Comments: Optimization by Building and Using Probabilistic Models. Workshop at the 2004 Genetic and Evolutionary Computation Conference

Report number: IlliGAL Report No. 2004020 ACM Class: G.1.6; G.3; I.2.6; I.2.8