CS4234
Optimization Algorithms
Welcome!
CS4234 Overview
 Optimization Algorithms
http://www.comp.nus.edu.sg/~gilbert/CS4234
 Instructor: Seth Gilbert
Office: COM2-323
Office hours: by appointment
Optimization Algorithms
Optimization:
Find the minimum/maximum…
Optimization Algorithms
Optimization:
Find the minimum/maximum:
Discrete: a collection of items
Optimization Algorithms
Optimization:
Find the minimum/maximum:
Discrete: a collection of items
Combinatorial: a collection of items generated
by counting, combining, and enumerating.
Optimization Algorithms
Optimization:
Find the minimum/maximum:
Discrete: a collection of items
Combinatorial: a collection of items generated
by counting, combining, and enumerating.
Examples:
Graphs
Matroids
Similar structures…
Combinatorial Optimization
Find the “best” item in a large set of items:
Problem
Set of items
Size
Difficulty
Searching
List of integers
Linear
Easy
Shortest paths
All paths in a graph
Exponential
Easy
Minimum spanning tree
All spanning trees
Exponential
Easy
Steiner tree
All steiner trees
Exponential
Hard
Travelling salesman
All possible tours
Exponential
Hard
Matching
All possible matchings Exponential
Easy
Bipartite vertex cover
All possible covers
Exponential
Easy
Vertex cover
All possible covers
Exponential
Hard
Maximum clique
All possible subsets
Exponential
Very Hard
Combinatorial Optimization
Find the “best” item in a large set of items:
Problem
Difficulty
Maintain student records
Easy
Data compression
Easy
Program halting problem
Impossible
VLSI chip layout
Hard
Exam timetable scheduling
Hard
Job assignment problem
Easy
Computer deadlock problem
Easy
Finding patterns in a database
Easy
Combinatorial Optimization
Operations Research:
How to make better decisions (e.g., maximize profit)
Project planning / critical path analysis
Facility location: where to open stores / plants
Floorplanning: layout of factory or computer chips
Supply chain management
Berth assignment problem (BAP): port management
Assignment problems (e.g., weapon target assignment)
Routing / transportation problems: buses, subways, trucking.
Airline ticket pricing
Optimization Algorithms
Optimization:
Find the minimum/maximum:
Discrete: a collection of items
Combinatorial: a collection of items generated
by counting, combining, and enumerating.
Continuous: given a function f(x), find the
vector x that maximizes f(x).
Optimization Algorithms
“If you need your software to run twice as fast,
hire better programmers.
But if you need your software to run more than
twice as fast, use a better algorithm.”
-- Software Lead at Microsoft
“... pleasure has probably been the main goal all
along.
But I hesitate to admit it, because computer
scientists want to maintain their image as hardworking individuals who deserve high salaries...
”
-- D. E. Knuth
“... pleasure has probably been the main goal all
along.
But I hesitate to admit it, because computer
scientists want to maintain their image as hardworking individuals who deserve high salaries...
”
-- D. E. Knuth
CS4234 : Optimization Algorithms
Brand new class:
– We can make this class what we want.
– Talk to me about your goals, interests, etc.
CS4234 : Optimization Algorithms
Brand new class:
– We can make this class what we want.
– Talk to me about your goals, interests, etc.
Where did this class come from?
– CS5234: Combinatorial and Graph Algorithms
– More general: all sorts of optimization (not only
graphs).
– More specific: just optimization
Optimization Algorithms
Target students:
– Advanced (3rd or 4th year) undergraduates
– Interested in algorithms
– Interested in tools for solving hard problems
Prerequisites:
– CS3230 (Analysis of Algorithms)
– Mathematical fundamentals
CS4234 Overview
 Mid-term exam
October 6
In class
 Final exam
November 25
Exams will be graded and returned.
CS4234 Overview
 Grading
40% Problem sets
25% Mid-term exam
35% Final exam
 Problem sets
– 5-6 sets (about every 1-2 weeks)
– Focused on algorithm design and analysis.
CS4234 Overview
 Mini-Project
Small project
Idea: put together some of the different
ideas we have used in the class.
Time scale: last 2-3 weeks of the semester.
CS4234 Overview
 Released today/tomorrow
Survey:
On IVLE.
What is your background?
Not more than 10 minutes.
PS1:
Released tomorrow.
CS4234 Overview
 Problem set grading
Simple scheme:
3 : excellent, perfect answer
2 : satisfactory, mostly right
1 : many mistakes / poorly written
0 : mostly wrong / not handed in
-1 : utter nonsense
CS4234 Overview
 What to submit:
Concise and precise answers:
Solutions should be rigorous, containing all
necessary detail, but no more.
Algorithm descriptions consist of:
1. Summary of results/claims.
2. Description of algorithm in English.
3. Pseudocode, if helpful.
4. Worked example of algorithm.
5. Diagram / picture.
6. Proof of correctness and performance
analysis.
CS4234 Overview
 How to draw pictures?
By hand:
Either submit hardcopy, or scan, or take a
picture with your phone!
Or use a tablet / iPad…
Digitally:
1.
2.
3.
4.
xfig (ugh)
OmniGraffle (mac)
Powerpoint (hmmm)
???
CS4234 Overview
 Policy on plagiarism:
Do your work yourself:
Your submission should be unique, unlike
anything else submitted, on the web, etc.
Discuss with other students:
1. Discuss general approach and techniques.
2. Do not take notes.
3. Spend 30 minutes on facebook (or equiv.).
4. Write up solution on your own.
5. List all collaborators.
Do not search for solutions on the web:
Use web to learn techniques and to review
material from class.
CS4234 Overview
 Policy on plagiarism:
Penalized severely:
First offense: minimum of one letter grade lost on
final grade for class (or referral to SoC disciplinary
committee).
Second offense: F for the class and/or referral to
SoC.
Do not copy/compare solutions!
Textbooks
Introduction to Algorithms
– Cormen, Leiserson, Rivest, Stein
– Recommended…
Textbooks
Algorithm Design
– Kleinberg and Tardos
– Recommended…
CS4234 Overview
 Topics (tentative, TBD)
Introduction to combinatorial optimization
Vertex cover, set cover, Steiner tree, TSP
Flows and matching
Maximum flow, bipartite matching
Linear programming
LPs, duality, relaxations, rounding
Continuous optimization
Gradient descent, meta-heuristics