OmeGA
A Competent Genetic Algorithm for
Solving Permutation and Scheduling Problems
Genetic Algorithms and
Evolutionary Computation
Consulting Editor, David E. Goldberg
University of Illinois at Urbana-Champaign
[email protected]
Additional titles in the series:
Efficient and Accurate Parallel Genetic Algorithms, Erick Cantu-Paz ISBN: 07923-7221-2
Estimation of Distribution Algorithms: A New Tool for Evolutionary
Computation, edited by Pedro Larraiiaga, Jose A. Lozano ISBN: 0-7923-7466-5
Evolutionary Optimization in Dynamic Environments, Jurgen Branke
7923-7631 -5
Anticipatory Learning Classifier Systems, Martin V. Butz
ISBN: 0-
ISBN: 0-7923-7630-7
Evolutionary Algorithms for Solving Multi-Objective Problems, Carlos A. Coello
Coello, David A. Van Veldhuizen, and Gary B. Lamont ISBN: 0-306-46762-3
Genetic Algorithms and Evolutionary Computation publishes research monographs, edited
collections, and graduate-level texts in this rapidly growing field. Primary areas of coverage include
the theory, implementation, and application of genetic algorithms (GAs), evolution strategies (ESs),
evolutionary programming (EP), learning classifier systems (LCSs) and other variants of genetiC and
evolutionary computation (GEC). Proposals in related fields
such as artificial life, adaptive behavior, artificial immune
systems, agent-based systems, neural computing, fuzzy
systems, and quantum computing will be considered for
GeneUc Algorithms and
publication in this series as long as GEC techniques are part
Evolutionary Computation
of or inspiration for the system being described. Manuscripts
describing GEe applications in all areas of engineering,
commerce, the sciences, and the humanities are encouraged. http://www.wkap.nVseries.htmlGENA
GENAGENAGENA
GENAGENAGENA
OmeGA
A Competent Genetic Algorithm for
Solving Permutation and Scheduling Problems
by
Dimitri Knjazew
SAP AG, Germany
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
Library of Congress Cataloging-in-Publication Data
Knjazew, Dimitri, 1974OmeGA: a competent genetic algorithm for solving permutation and scheduling
problems / by Dimitri Knjazew.
p. cm. - (Genetic algorithms and evolutionary computation ; 6)
Inc1udes bibliographical references and index.
ISBN 978-1-4613-5249-5
ISBN 978-1-4615-0807-6 (eBook)
DOI 10.1007/978-1-4615-0807-6
1. Genetic algorithms. 2. Permutations. 3. Scheduling-Mathematics. 1. Title. II.
Series.
QA402.5.K595 2002
5 I 9.3--dc2 I
2001050789
Copyright © 2002 by Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 2002
Softcover reprint ofthe hardcover Ist edition 2002
AlI rights reserved. No part ofthis publication may be reproduced, stored in a retrieval
system or transmitted in any form or by any means, mechanical, photo-copying, recording, or
otherwise, without the prior written permission ofthe publisher Springer Science+Business
Media, LLC.
Printed on acid-free paper.
The Publisher offers discounts on this book for course use and bulk purchases.
For further information, send email to<[email protected]>
For Mom, Dad, and my
brothers Alexander and
Andrej
Contents
List of Figures
List of Tables
Foreword
Preface
Acknowledgments
Introduction
1.1
1.2
1.3
Building Blocks, Deceptive Problems,and Linkage Problem
Competent Genetic Algorithms
Organization of the Book
1. DEVELOPMENT OF THE OMEGA
1.1 The Mechanics of the Fast Messy GA
1.2 Using Random Keys for Representation
1.3 Designing the OmeGA
1.4 Ordering Deceptive Problems
1.5 Problem Codings
1.6 Experiments
1.7 Summary
2. PERFORMANCE ANALYSIS OF THE OMEGA
2.1
2.2
2.3
2.4
2.5
2.6
Scale-up Analysis
New Ordering Deceptive Problems
Tests with Uniform and Nonuniform Scaling
Test with Nonuniform Building-Block Size
Tests with Overlapping Building Blocks
Summary
ix
xi
xiii
xv
xvii
xix
xix
xx
xxi
11
13
14
18
19
24
27
28
33
40
42
44
47
viii
OMEGA: A COMPETENT GENETIC ALGORITHM
3. APPLICATION TO A SCHEDULING PROBLEM
3.1 Introduction to Scheduling Problems
3.2 Problem Formulation
3.3 Schedule Representation and Decoding
3.4 Experiments
3.5 Summary
51
51
55
58
62
68
4. CONCLUSIONS AND FUTURE WORK
69
Appendices
Appendix A:
Appendix B:
AppendixC:
71
71
The Benchmark Input Data
Best Schedules
Source Code of OmeGA
79
89
References
147
Index
151
List of Figures
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
1.10
1.11
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
3.1
3.2
3.3
Usage of competitive templates
Cut and splice operators
Building-block disruption and preservation
Building-block filtering step
Overall chromosome length during filtering phase
Ladder of deception
Multiple epochs in the OmeGA
Relative ordering problem with deflen6 coding
Absolute ordering problem with deflen6 coding
Relative ordering problem with loose coding
Absolute ordering problem with loose coding
Function evaluations versus population size for OmeGA
Epochs versus population size for OmeGA
Population sizing curves
Scale-up curve of the OmeGA
Economic population sizes and epochs
Population sizing for the RKGA
Scale-up behavior of the OmeGA and the RKGA
Correct subfunctions and fitness
Correct subfunctions, nonuniform scaling
Correct subfunctions, nonuniform size
Correct subfunctions, 4-chains-overlap problem
Correct subfunctions, full-chain-overlap problem
Sample schedule
Plant model
Schedule for shipment S 15
2
3
4
7
8
10
14
21
22
23
24
31
32
33
34
35
36
37
40
41
43
47
48
54
55
56
x
OMEGA: A COMPETENT GENETIC ALGORITHM
3.4
3.5
3.6
3.7
3.8
3.9
3.10
3.11
3.12
3.13
Recipe for shipment Sl5
Chromosome structure for BENCHOI problem
Left chromosome part
Right chromosome part
Encoded schedule
Average makespan versus population size
Average makespan versus function evaluations
Average makespan versus function evaluations
Convergence plots
Experimental results of multicriteria optimization
57
58
59
61
62
63
64
65
66
67
List of Tables
1.1
1.2
1.3
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
3.1
A.I
A.2
A.3
A.4
A.5
A.6
A.7
Search spaces of ordering deceptive problems
Problem codings
Performance of crossover operators
Paramters sizes used in the OmeGA for scale-up tests.
Loose coding, frel problem, 1 = 64
Best and the worst relative ordering schemata
Loose coding, f~K problem, 1 = 50
Tight coding, mixed-size problem, 1 = 45
4-chains-overlap coding, 1 = 40
Full-chain-overlap coding. 1 = 40
Building-block pattern
Best results by the OmeGA and the RKGA
Machine speed factors
Products and product families
Transition times
Operations
Operations
Product ingredients
Shipments
17
18
19
29
30
39
40
42
44
45
49
66
71
72
73
74
75
76
77
Foreword
Researchers and practitioners alike are increasingly turning to search,
optimization, and machine-learning procedures based on natural selection and
natural genetics to solve problems across the spectrum of human endeavor.
These genetic algorithms and techniques of evolutionary computation are
solving problems and inventing new hardware and software that rival human
The Kluwer Series on Genetic Algorithms and Evolutionary
designs.
Computation publishes research monographs, edited collections, and graduatelevel texts in this rapidly growing field. Primary areas of coverage include the
theory, implementation, and application of genetic algorithms (GAs), evolution
strategies (ESs), evolutionary programming (EP), learning classifier systems
(LCSs) and other variants of genetic and evolutionary computation (GEC). The
series also publishes texts in related fields such as artificial life, adaptive
behavior, artificial immune systems, agent-based systems, neural computing,
fuzzy systems, and quantum computing as long as GEC techniques are part of or
inspiration for the system being described.
Dimitri Knjazew's book is the most recent addition to the series, and it is a nice
"recombination" of two increasingly important areas in GA implementation and
practice. Knjazew's system called OmeGA or the ordering messy genetic
algorithm combines some of the latest in competent GA technology to solve
scheduling and other permutation problems. Competent GAs are those designed
for principled solutions of hard problems, quickly, reliably, and accurately.
Permutation and scheduling problems are difficult combinatorial optimization
problems with commercial import across a variety of industries. Knjazew
approaches his twin subjects systematically and clearly. The first part of the
book presents the clearest description of messy GAs written to date along with
Knjazew's innovative adaptation of the method to ordering problems. The
second part of the book investigates the algorithm on boundedly difficult test
functions, showing principled scale up as problems become harder and longer.
Finally, the book applies the algorithm to a test function drawn from the
literature of scheduling. I wholeheartedly recommend this book to readers
interested in fast solutions to permutation and scheduling problems or to those
who are more generally interested in understanding how to design genetic
algorithms that work fast and scale well.
David E. Goldberg, Consulting Editor
University of Illinois at Urbana-Champaign
[email protected]
Preface
This book introduces the reader to a new generation of genetic algorithms
(GAs) - the so-called competent GAs. In particular, it focuses on one representative which is specialized for solving permutation and scheduling (production planning) problems.
Over the past few years, many different genetic and evolutionary algorithms
have been designed and applied to a wide variety of combinatorial problems.
Research in this area has been constantly growing, since there exist many commercially important applications.
Unfortunately, the scalability of such algorithms has been very poorly investigated. Since many methods use ad hoc techniques and operators, they do not
scale up. However. typical industrial problems are often so large and complex
that traditional optimization methods are expected to fail or yield inacceptable
solutions.
The purpose of this book is to develop a permutation-oriented competent
GA and to demonstrate its performance and scalability on hard permutation
problems. Moreover. the algorithm is applied to a real world scheduling problem that has been used as a standard benchmark within the German company
SAP-one of the leading enterprise resource planning software vendors.
The results of this research suggest that in the near future competent genetic algorithms could be used as reliable problem solvers in many various
fields including structural optimization, parameter optimization, scheduling,
timetabling. and route planning. The book addresses readers with a basic
knowledge of genetic algorithms. Especially the GA expert should find many
techniques and advices that might be helpful for his research. The book could
also be used in classes on genetic and evolutionary computation, as well as in
operations research.
xvi
OMEGA: A COMPETENT GENETIC ALGORITHM
The introduction of the book provides the reader with background information about competent genetic algorithms. The first chapter develops the
ordering messy genetic algorithm (OmeGA) and the second chapter presents a
detailed scalability and performance analysis of the method. In the third chapter the OmeGA is applied to the SAP scheduling benchmark mentioned above.
Conclusions and suggestions for future research are given in the last chapter.
The appendices contain the problem data, some detailed optimization results,
and the source code of the OmeGA in C++.
Acknowledgments
First, I wish to thank my advisor, supporter, and mentor, Professor David E.
Goldberg. Without his guidance, advice and encouragement I would never be
able to complete this book. I greatly appreciate his patience with me during
my stay at the University of Illinois.
I would like to give special thanks to Professor Hans-Paul Schwefel for encouraging me to spend a portion of my diploma thesis studies with David E.
Goldberg, thus enabling this collaboration to take place. Professor Schwefel
also provided important guidance and essential suggestions that greatly improved this work.
The Illinois Genetic Algorithms Laboratory (IlliGAL) is an excellent place
for research and study. I would like to thank Franz Rothlauf, Martin Pelikan,
Erick Cantu-Paz, Fernando Lobo, Martin Butz, Hillol Kargupta, Kumara Sastry, Laura Albert, and Jacob Borgerson for their useful comments and suggestions.
I greatly acknowledge the help of Heinrich Braun from the German company SAP. He provided me with information and material, that formed the
basis for a large portion of this book. I further acknowledge the support of the
"Studienstiftung des Deutschen Volkes"(Germany) and the Sigma-Xi Research
Society.
Finally, I thank my parents and my two brothers Andrej and Alexander for
their inspiration and understanding.
The work was sponsored by the Air Force Office of Scientific Research, Air
Force Materiel Command, USAF, under grant F49620-00-1-0163. Research
funding for this work was also provided by the National Science Foundation
under grant DMI-9908252. Support was also provided by a grant from the U.
S. Army Research Laboratory under the Federated Laboratory Program, Coop-
xviii
OMEGA: A COMPETENT GENETIC ALGORITHM
erative Agreement DAALOl-96-2-0003. The U. S. Government is authorized
to reproduce and distribute reprints for Government purposes notwithstanding
any copyright notation thereon.
The views and conclusions contained herein are those of the authors and
should not be interpreted as necessarily representing the official policies or
endorsements, either expressed or implied, of the Air Force Office of Scientific Research, the National Science Foundation, the U. S. Army, or the U. S.
Government.
Introduction
Numerous genetic and evolutionary algorithms have been designed for
a wide variety of permutation problems over the past few years. Problems
such as the traveling salesman problem (Goldberg & Lingle, 1985), scheduling (Davis, 1985), vehicle route planning (Blanton Jr. & Wainwright, 1993)
or integrated circuit design (Louis & Rawlins, 1991) have been tackled. Research in this area is very interesting, since there are many permutation-based
commercially important applications. Unfortunately, many methods use either
problem-specific or ad-hoc representation codings and operators. Also, only
little research has been done to examine how permutation-solving genetic and
evolutionary algorithms scale up or, in other words, how their computational
complexity increases with the underlying problem difficulty and size.
Therefore, it would be interesting to design genetic algorithms (GAs) that
use efficient codings and operators and have good scale-up properties. Furthermore, a more detailed and systematic performance analysis should be done. We
suggest that so-called competent genetic algorithms which solve hard problems
quickly, reliably, and accurately (Goldberg, 1993), would be good approaches
for this undertaking.
This chapter gives background information necessary for understanding the
forthcoming chapters. First, we discuss the notion of building blocks, deceptive problems, and the linkage problem. Then, we give a short overview of
competent GAs and describe the organization of the book.
1.1
Building Blocks, Deceptive Problems, and the Linkage
Problem
One approach to investigate the scale-up behavior of a GA is testing it on
artificial problems where the solution is known a priori and where the problem
difficulty can be varied. For this purpose researchers in the GA community
xx
OMEGA: A COMPETENT GENETIC ALGORITHM
frequently used deceptive problems, which are hard multimodal optimization
problems for binary strings introduced by Goldberg (1987). A deceptive problem may be designed by combining a desired number of deceptive subfunctions
that mislead the genetic algorithm letting it converge to certain local optimal
points. No information about the subfunctions is passed to the GA. To find
the global optimum, partial solutions (schemata) with above-average fitnessthe so-called building blocks (BBs)-must be identified and grouped together.
Kargupta, Deb, and Goldberg (1992) developed ordering deceptive problems
for permutations which we will discuss in detail in section 1.4.
A deceptive problem's degree of difficulty grows with increasing number
and size (order) of the subfunctions. Particularly for the simple genetic algorithm, the problem difficulty increases when a loose coding is chosen such
that the elements of the subfunctions are mapped to distant positions within
the problem representation. This results in a greater building-block length and,
consequently, the BBs are more likely to be disrupted when traditional recombination operators such as single-point or two-point crossover are applied. In
general, no fixed recombination operators guarantee proper mixing with an arbitrary coding. This problem is usually referred to as the linkage problem in the
literature. Thierens and Goldberg (1993) showed in a dimensional analysis of
mixing processes in simple genetic algorithms that the required population size
grows exponentially with the building-block length and number. Section 1.5
introduces some problem codings for ordering deceptive problems.
1.2
Competent Genetic Algorithms
To tackle the linkage problem and to achieve a better scale-up behavior, firstgeneration genetic algorithms need to be extended by some additional mechanisms. Various competent genetic algorithms mentioned above have been
developed for this purpose. They can be divfded into two classes:
1 methods based on evolving the representation of solutions or adapting recombination operators among individual solutions
2 methods based on extracting information from the entire set of promising
solutions to generate new solutions
Examples of the first class are the fast messy genetic algorithm (Goldberg,
Deb, Kargupta, & Harik, 1993), the gene expression messy genetic algorithm
(Bandyopadhyay, Kargupta, & Wang, 1998), and the linkage learning genetic
algorithm (Harik, 1997). Approaches belonging to the second class are the
extended compact genetic algorithm (Harik, 1999)and the Bayesian optimization algorithm (Pelikan, Goldberg, & Cantu-Paz, 1999). In this book we focus
INTRODUCTION
xxi
on the fast messy GA (fmGA) because it can be easily transformed into a
permutation-solving GA.
1.3
Organization of the Book
This book consists of three main parts:
• development of a permutation-oriented competent GA
• performance analysis of the competent GA
• application to a scheduling problem.
In the first chapter we design a competent genetic algorithm that is based on a
fast messy GA and is specialized for sequencing problems. Also, its capability
of solving difficult permutation problems is tested in pilot experiments. In the
second chapter we present a detailed scalability analysis of the algorithm and
test it on hard sequencing problems. The third chapter applies the GA to a
real world scheduling problem and demonstrates its performance. Throughout
the book we compare the GA's convergence behavior to the simple GA (Holland, 1975). Chapter 4 concludes the book and gives suggestions for future
research. The appendices A and B present the problem data and two solutions
of a scheduling problem considered in this book. In the appendix C the C++
code of the ordering messy GA is listed.
Chapter 1
DEVELOPMENT OF THE ORDERING
MESSY GENETIC ALGORITHM
This chapter develops the ordering messy GA (OmeGA), a fast messy GA
(fmGA) specialized for permutation problems. It represents the solutions by
vectors of real numbers-the so-called random keys introduced by Bean (1994).
In a number of experiments it is shown that the OmeGA significantly outperforms the simple GA in solving ordering deceptive problems, which are hard
sequencing problems defined elsewhere (Kargupta et aI., 1992).
We start with an introduction to the fast messy genetic algorithm (section 1.1).
We then explain the concept of random keys (section 1.2) and demonstrate how
they can be incorporated into the fmGA to represent permutations. Section 1.3
develops the OmeGA. Thereafter, we describe ordering deceptive problems
(section 1.4) and different codings that determine the problem difficulty (section 1.5). Finally, experimental results are presented (section 1.6) and the chapter is summarized (section 1.7).
1.1
The Mechanics of the Fast Messy GA
The first steps towards the development of competent genetic algorithms
were taken by Goldberg, Korb, and Deb (1989) who developed the messy GA
in 1989. The messy GA formed a basis for the fast messy GA developed
four years later by Goldberg et al. (1993). However, first-generation messy
GAs suffered some handicaps which made application to large-sized problems
impractical. These were, among others, the lack of implicit parallelism and
large memory consumption during the initialization phase, often referred to
as the initialization bottleneck. An interested reader might refer to Goldberg,
Deb, and Korb (1990) for more details.
This section briefly describes key features of the fmGA, how it works and
how the initialization bottleneck has been overcome therein, making the messy
GA "fast". We first explain the messy representation and operators. Then,
D. Knjazew, OmeGA
© Kluwer Academic Publishers 2002
2
OMEGA: A COMPETENT GENETIC ALGORITHM
messy chromosome:
(30)(1 1)(50)(7 1)
filled chromosome:
template:
t t t ttt
1011100101
Figure 1.1. Usage of competitive templates. Underspecified messy chromosomes are evaluated by taking the missing genes from the template.
we look at the organization of the fmGA and finish with some important techniques that contribute to its success. For a more detailed description we refer
to Goldberg et al. (1993).
1.1.1
Messy Representation
In messy genetic algorithms the genes of a chromosome are represented by
the pair (allele locus, allele value), such as, for example, the two messy chromosomes
«3 0) (10) (2 1) (4 1) (5 0)) and
«2 1) (5 0) (3 0) (1 0) (4 1)),
which are both equivalent to the binary string 01010. The chromosomes may
have different lengths. For instance, the chromosomes
A: « 5 1) (1 1) (3 0) (1 0) (4 1) (3 1) (2 1) (4 0) (5 0)) and
B: «3 0) (1 1) (5 0))
both represent valid strings of length five. To evaluate over-specified chromosomes like example A, the genes are scanned from left to right with a firstcome-first-serve precedence rule. Thus, example A would encode the bit string
11011. For evaluating underspecified chromosomes, like example B, a competitive template which is a completely specified fixed-bit string is used in the
fmGA. Before evaluation, the chromosome's missing genes are filled with the