Introduction to
Algorithms
Jiafen Liu
Sept. 2013
About me
• Teacher：Liu Jiafen
• Mail: [email protected]
• Research Interest:
– Information security
– Formal Method and Verification of
Correctness
– Data Mining and Business Intelligence
– Development of soft under iOS
• Homepage:
– http://it.swufe.edu.cn/201109/24/201109241343095261.html
Now it’s your turn
• Please introduce yourself
– Name
– Age
– Graduated School
– Specialty and Interest
• Background Investigation
Have you studied data structure and other prerequisite
courses of CS?
Have you programmed in practice before?
About the Course
• Site:
– http://fife.swufe.edu.cn/BILab/lectures/algo/al
go.htmlLecture
– Notes can be found here.
• Related Resources can also be found on
http://ocw.mit.edu/courses, you can also
watch lectures online
• No Textbook
About the Course
• 16 weeks (the last 2 is reserved for exam)
• Site: Tongbo Building 206
• Time: Can we choose another proper time?
– Tuesday afternoon（5-7 or 6-8）?
– Tuesday night（10-12）?
– Thursday night（10-12）
– Or we have to split into 2 period(first 2 in
classroom, and last 1 on computers)?
Grading
• 20%
Attendance
• 50%
Performance
– 1 presentation at least (10%)
– assignments (30%)
• 30% Final paper
Course Objectives
• What is an algorithm?
• What’s the difference between algorithm
and program?
• Why we need this course?
• How to evaluate a algorithm?
• How can we design a good algorithm?
• How to make an algorithm implemented?
What is an algorithm?
• Algorithm is a problem-solving method or
process.
• It helps us to understand scalability.
• Algorithm satisfy the following properties:
– Input
– Output
– Unambiguous
– Finite
– Feasible
Algorithm and Program
• Program is an implementation of algorithm
using a programming language.
• Algorithm design takes the most important
place in Software Engineering.
• Programmer VS Coder?
• How to become an excellent programmer?
What is important for a program?
correctness
robustness
functionality
maintainablity
simplicity
Performance
userfriendliness
scalability
Why we need performance?
• Performance is the currency of computing.
• Example: C family VS Java
• This course makes theoretical study of
computer program performance and
resource usage.
– How to evaluate an algorithm?
– How can we design a good algorithm?
– How to make an algorithm implemented?
The problem of sorting
• Input: sequence 〈a1, a2, …, an〉 of numbers.
• Output: permutation 〈a1', a2', …, an'〉 such
that a1' ≤ a2' ≤…≤ an'.
• Example:
Input : 8 2 4 9 3 6
Output: 2 3 4 6 8 9
• How can we do that?
– Recall the process of playing cards.
Insertion sort
•
8
2
4
9
3
6
Insertion sort
•
8
2
4
9
3
6
•
2
8
4
9
3
6
Insertion sort
•
8
2
4
9
3
6
•
2
8
4
9
3
6
•
2
4
8
9
3
6
Insertion sort
•
8
2
4
9
3
6
•
2
8
4
9
3
6
•
2
4
8
9
3
6
•
2
4
8
9
3
6
Insertion sort
•
8
2
4
9
3
6
•
2
8
4
9
3
6
•
2
4
8
9
3
6
•
2
4
8
9
3
6
•
2
3
4
8
9
6
Insertion sort
•
8
2
4
9
3
6
•
2
8
4
9
3
6
•
2
4
8
9
3
6
•
2
4
8
9
3
6
•
2
3
4
8
9
6
•
2
3
4
6
8
9
Done!
Insertion sort
Running time
• The running time depends on the input: an
already sorted sequence is easier to sort.
• Since short sequences are easier to sort
than long ones, we will parameterize the
running time by the size of the input.
Kinds of analyses
• Worst-case: (usually)
– T(n) =maximum time of algorithm on any
input of size n.
• Average-case: (sometimes)
– T(n) =expected time of algorithm over all
inputs of size n.
– Need assumption of statistical distribution of
inputs.
• Best-case: (bogus)
– Cheat with a slow algorithm that works fast
on some input.
Machine-independent time
• Generally, we seek upper bounds on the
running time, Why?
– Because everybody likes a guarantee.
• What is insertion sort’s worst-case time?
• It depends on the speed of our
computer:
– relative speed (on the same machine)
– absolute speed (on different machines)
Asymptotic Analysis
Big Idea
• Ignore machine-dependent constants.
• Look at growth of T(n) as n→∞.
Θ-notation
• Math:
Θ(g(n)) = { f(n) : there exist positive
constants c1, c2, and n0 such that 0 ≤ c1g(n)
≤ f (n) ≤ c2g(n) for all n ≥ n0}
• Engineering:
Drop low-order terms;
Ignore leading constants.
• Example: 3n3 + 90n2–5n+ 6046
= Θ(n3)
Θ-notation
• Θ-notation is fabulous because it satisfies
our issue of being able to compare both
relative and absolute speed.
Asymptotic performance
• When n gets large enough, a Θ (n2) algorithm
always beats a Θ(n3) algorithm.
• We shouldn’t ignore asymptotically slower algorithms, however.
• Real-world design situations often call for a careful balancing of
engineering objectives.
• Asymptotic analysis is a useful tool to help to structure our thinking.
Insertion sort analysis
• Assume that every elemental operation is
going to take some constant amount of time.
• Worst case : Input reverse sorted.
[arithmetic series]
• Average case: All permutations equally
likely.
About Θ-notation
• It is more of a descriptive notation than it
is a manipulative notation.
• Is insertion sort a fast sorting algorithm?
– Moderately so, for small n.
– Not at all, for large n.
Another method : Merge sort
• Merging two sorted arrays
20
13
20
13
12
12
11
11
9
7
9
7
2
1
• Time = Θ(n) to merge a total of n
elements (linear time).
2
1
Merge Sort
T(n)
Θ(1)
2T(n/2)
Θ(n)
Θ(1) : Abuse, but we can ignore it.
2T(n/2) :Should be
, but it
turns out not to matter asymptotically.
Cost for merge sort
• We shall usually omit stating the base
case when T(n) = Θ(1) because it has no
effect on the asymptotic solution to the
recurrence.
• Lecture 2 will provide several ways to find
a good upper bound on T(n), for example,
Recursion Tree.
Recursion Tree
• We can write as T(n) = 2T(n/2) + cn, where
c > 0 and it is constant.
T(n)
Recursion Tree
• We can write as T(n) = 2T(n/2) + cn, where
c > 0 and it is constant.
cn
T(n/2)
T(n/2)
Recursion Tree
• We can write as T(n) = 2T(n/2) + cn, where
c > 0 and it is constant.
cn
cn/2
T(n/4)
cn/2
T(n/4) T(n/4)
T(n/4)
Recursion Tree
• We can write as T(n) = 2T(n/2) + cn, where
c > 0 and it is constant.
cn
cn
cn/2
Height
=
logn+1
？
Θ(1)
T(n/4)
cn
cn/2
T(n/4) T(n/4)
T(n/4)
cn
……
#(leaves) = n?
Θ(n)
Total = cn*logn+Θ(n)
Θ(n logn)
Conclusion
• Θ(nlgn) grows more slowly than Θ(n2) .
• Therefore, merge sort asymptotically
beats insertion sort in the worst case.
• In practice, merge sort beats insertion sort
for n > 30 or so.
• Go test it out for yourself!
Homework
• Read Chapter1 and Chapter 2.
• Recall another sort method “Bubble sort”
and express with pseudocode, then
compute its cost using Θ-notation (Hand
in paper edition the next class).