ECE733
Nonlinear Optimization for Electrical Engineers
Dr. Mohamed Bakr,
905 525 9140 x24079
[email protected]
Lecture 0
0-1
Info About Myself
B.Sc. in Electronics and Communication Engineering, Cairo
University, Cairo, Egypt with Distinction (honors), 1992
M.Sc. in Engineering Mathematics (Optimization), Cairo
University, 1996
Ph.D. in Computer Aided Design (CAD) of Microwave Circuits,
McMaster University, 2000
P.Eng., Ontario, 2003
Full professor, 2013
Author/CoAuthor of over 200 journal and conference papers,
one book, two book chapters, and two patents
Lecture 0
0-2
Info About Myself (Cont’d)
Research Areas: Optimization methods, computer-aided design
and modeling of microwave circuits, neural networks
applications, computational electromagnetics, and nanophotonics
Awards/Scholarships:
TRIO Student Internship in OSA, inc. 1997
Ontario Graduate Scholarship (OGS) 1998-2000,
NSERC PostDoctoral Fellowship 2000-2001,
Premier’s Research Excellence Award (PREA) 2003-2009
McMaster Tenure 2007
Sabbatical Leave with RIM (2008-2009)
NSERC Accelerator Supplement Award (DAS), 2011
Supervisor/Co-supervisor to a number of graduate students
Lecture 0
0-3
Teaching Experience
Teaching Assistant in Engineering Mathematics (Cairo
University), 1992-1996
Teaching Assistant in Electrical Engineering (McMaster
University) 1996-1999
Assistant Professor in the Department of Electrical and
Computer Engineering, McMaster University 2002-2007:
 ECE 750 Advanced Engineering Electromagnetics
ECE 2EI4 Electronic Devices and Circuits
ECE 3TP4 Signals and Systems
ECE 757 Numerical Techniques in Electromagnetics
ECE 2EI5 Electronic Devices and Circuits
ECE 3FI4 Theory and Applications in Electromagnetics
Lecture 0
0-4
Teaching Experience (Cont’d)
ECE 2FH3 Electromagnetics I
ECE 2CI5 Introduction To Electrical Engineering
ECE 3FK4 Electromagnetics II
ECE 4OI6 Engineering Design
ECE 718 Nonlinear Optimization
Developer of a number of coursewares for several courses
Lecture 0
0-5
Course
Overview
1-Introduction To Vector Analysis and Optimization
Introductory mathematical tools
Historical background
Jargon of optimization problems and their classifications
Lecture 0
0-6
Course Overview (Cont’d)
2-Classical Optimization Approaches
Single-variable methods
Multi-variable methods
The KKT conditions for equality and inequality constraints
Lecture 0
0-7
Course Overview (Cont’d)
3-One Dimensional Search Techniques
Why one-dimensional search is so important?
Derivative-free methods
Gradient and Hessian based methods
Lecture 0
0-8
Course Overview (Cont’d)
4-Unconstrained Optimization
Derivative-free approaches
Gradient-based techniques
Second-order methods
Lecture 0
0-9
Course Overview (Cont’d)
5-Constrained Optimization
Quadratic programming
Sequential quadratic programming
Penalty methods
Gradient projection methods
Methods of feasible direction
Lecture 0
0-10
Course Overview (Cont’d)
6. Global Optimization Methods
Old population
New population
Simulated annealing
Genetic algorithms
Particle swarm optimization
Lecture 0
0-11
Course Overview (Cont’d)
7. Space Mapping Optimization
design
parameters
fine
model
responses
surrogate
space
mapping
space
mapping
input
mapping
coarse
model
implicit
mapping
space
mapping
responses
output
mapping
Aggressive space mapping
Trust region space mapping
Implicit space mapping
Surrogate-based space mapping
Lecture 0
0-12
Course Overview (Cont’d)
8. Adjoint Sensitivities and Their Applications
x
Original
Simulation
R
x
Adjoint
Simulation
Rˆ
Using only at most one extra simulation, the sensitivities of the
response with respect to all design parameters are obtained
This makes gradient-based optimization far more efficient
Lecture 0
0-13
Course Overview (Cont’d)
Text: Engineering Optimization Theory and Practice, Singiresu
S. Rao, Third Edition
or
Text: Nonlinear Optimization in Electrical Engineering with
Applications in MATLAB, Mohamed H. Bakr, IET Press, 2013
CLASSES: TBD
Course Webpage:
http://www.ece.mcmaster.ca/faculty/bakr/
ECE733/ECE733_Main_2014.htm
4 Matlab assignments and one final project are required
Lecture 0
0-14
Detailed Course Outline
Date Lecture
0
1
2
3
4
5
6
7
8
Lecture 0
Description
Course Outline
Introduction: Historical Background, statement of
optimization problem
Introduction: Classifications of Optimization problems
Classical Optimization Methods: single variable
optimization, unconstrained multivariate optimization
Equality Constraints: Solution by Direct substitution,
Method of constrained variation
Equality Constriants: Method of Lagrange multipliers
Inequality constraints: Kuhn-Tucker Conditions,
Constraint qualification
One Dimensional Search: why one dimensional search?,
Search with Fixed Step Size, Search with Accelerated Step
size
One Dimensional Search: Interval halving Method,
Fibonacci Method, Golden Section Search
0-15
9
10
11
12
13
14
15
16
17
18
19
Lecture 0
One Dimensional Search: Interpolation Methods, Newton
Method
One Dimensional Search: Quasi-Newton Method, Secant
Method, Practical Consideration
Unconstrained Nonlinear Optimization: Introduction and
basic concepts
Direct Search Methods: Random Walks, Grid Search,
Univariate Method, Simplex Method
Conjugate Gradient Methods: Powell’s Method, Conjugate
Directions
Indirect Methods: Steepest Descent, Conjugate Gradients
2nd Order Methods: Newton Method, Marquardt Method,
and Quasi Newton Methods
2nd Order Methods (Cont’d): The DFP formula, the BFGS
formula, summary
Constrained Nonlinear Optimization: Introduction and
basic concepts
Some Constrained Optimization Methods: Zoutendijk’s
method of feasible directions
Constrained Optimization (Cont’d): Rosen’s Gradient
0-16
20
21
22
23
24
25
26
27
28
Lecture 0
projection Method, sequential quadratic programming
Constrained Optimization (Cont’d): Penalty Methods
Global Optimization Techniques: Genetic Algorithms
Global Optimization Techniques (Cont’d): Simulated
annealing
Global Optimization Techniques(Cont’d): Particle Swarm
Optimization
Space Mapping Optimization and Modelling: Basic
Concepts, classical Space Mapping, Aggressive Space
Mapping
Space Mapping (Cont’d): surrogate-based optimization,
Output Space Mapping
Adjoint Variable Methods: The Frequency Domain Case
Adjoint Variable Methods: The Dynamic Case
Areas of Research in Optimization
0-17
General Comments
Lecture is divided into two parts each for about 1.0 Hr to
1.25 Hr. We will have a break in the middle
We will not focus on theorem proving. We will give a proof
as long as it is concise and useful
Engineering Applications will be given as much as possible
We will write all our optimization code. Ready functions in
packages will only be used for comparison
Material will be posted on the course webpage the day
before. Copy only examples not in the slides.
Lecture 0
0-18