Theoretical Computer Science I
Jian Liu
Institute for Theoretical Computer Science
Graz University of Technology
2017.03.10
Course information
2 SSt VO: Fr 11:15-12:45
1 SSt KU: Fr 13:15-14:00
Theoretische Informatik I
(2 VO 708.243, 1 KU 708.244):
Please register to the correct VO and KU
Grades
VO: written exam
 max. 100 Points
 4 exams per year
 New system to me
 Topics are consistent with those in
previous years
 Understanding the materials given in the
lectures and assignments
Grades
KU: written assignments
 Group work allowed (max. 3 each group)
 One report per group
 100-point grading
 Extra bonus (∗) points with challenge problems
 DIY required, copy not allowed
 Help you to understand the materials
Grades
KU: written assignments
 Report submitted online
 You will be enrolled into the online course system
 A few days later, you will get an email with login
information
 Assignments read by tutors
Course information
 Homepage: https://courses-igi.tugraz.at/courses/7
 Newsgroup:
news://news.tu-graz.ac.at:119/tu-graz.lv.ti1
 Skriptum/Lecture note: LATEX-project, from
previous students
- digital copy on the homepage
- contributions to it are possible (English?)
 Check the course website for more information
 Announcement from time to time
Textbooks
 M. Sipser, Introduction to the Theory of Computation,
PWS Publishing, Boston, 2012
 Ch. Papadimitriou, Computational complexity, AddisonWesley, 1994
 S. Aroa and B. Barak, Complexity Theory: A Modern
Approach. Cambridge University Press, 2009.
- a draft of this book is available for free on the Web.
 O. Goldreich, Computational Complexity: A Conceptual
Perspective, Weizmann Institute. - a draft of this book is available for free on the Web.
 Course Skriptum accumulated over years by previous
lectures and students.
Course topics
• Intuitive and formal computability
• Turing machines
• Time complexity, space complexity
• Complexity classes P, NP, ...
• Complete problems
• Randomized algorithms
• Some state-of-the-art research topics
(academic and industrial areas)
Today
 Introduction
Questions to Theoretical Computer Science
Computability
 Formal concept of Computability
Problems in graphs
Graph Reachability
 Examine complexity (example: REACH)
Landau Notation
Complexity of REACH
 Summary
Motivation
 There is a question/problem you want to solve
(making a phone book for the family).
 Find a easy method without lots of searching (writing
them down in a random order).
 It works for a small set of data (one family).
 In reality, the dataset is much bigger (the whole city).
 Code/algorithm is running (write it down every time?)
 Fast algorithm (need some time to think it about )?
 ... ...
Some fundamantal questions
in Theoretical Computer Science
 What are the basic abilities of a computer?
 What are the limitations of a computer?
Something more concrete:
 What is a problem?
 What is an algorithm?
 Is there an algorithm for a specific problem?
 Why are some problems difficult to solve?
 Is a solvable problem also efficiently solvable?
 What is efficiency?
What is a problem?
 Optimize the delivery route of a delivery
service!
 How about the weather tomorrow?
 For
, are there integer solutions
with n ≥ 3 (Fermat’s Last Theorem)?
 The computer is not responding. Does it make
sense to wait?
Computability
Each of us has a certain idea of what are
computable problems and whether they are
easy or difficult.
Sorting is relatively simple.
Basic operation (+,−,∗,/) is easier.
Mathematical proofs could be hard (Fermat’s
Last Theorem was proposed in 1637, and
solved by Andrew Wiles in 1995).
Formal concept of Computability
We are looking for a formal concept of
computability that ...
 precisely defined
 conceptually as simple and intuitive
 regardless of specific constraints such as
hardware, programming languages ...
Formal computable problems
First, we need a formal term for computable
problems!
Here we limit ourselves to:
 Problems in Graphs
 Problems in Boolean Circuits
 Formal Languages
Graph
Definition (Graph)
A graph G is a tuple (V , E), where V is a set of
nodes and E is a set of edges.
Definition (Edge)
An edge E is a tuple (v1, v2) of nodes v1, v2 ∈ V,
where v1 is the outgoing and v2 is the incoming
node.
Graph
 Directed Graph: G=(V, E)
 Node: V = {1, 2, 3, 4, 5}
 Edge:
E = { (1, 2), (1, 3), (2, 3),
(3, 1),(3, 5), (4, 3), (4, 5) }
Number of nodes: |V|
Number of edges: |E|
Graph Reachability
(REACH or PATH)
Given a directed graph (without multiple edges) G = (V, E)
with n nodes (n = |V|).
Is there a path from node 1 to node n?
 Decision problem (‘yes’/‘no’)
 REACH is a basic problem of theoretical computer
science,
because:
• Each solvable decision problem can be interpreted as a
directed graph.
• Many possible solution algorithms
Decision problems (Y/N)
There are several ways to solve the problem:
 Yes/No, path exists / does not exit
 List all paths
 Give a specific path (shortest)
In complexity theory, we consider only decision problems
 Reason: statement should be independent of the
encoder of the answer
 No restriction! All relevant problems can be a decision
problem
Solution-algorithm for REACH
Idea for (search algorithm):
Start at node 1.
Run recursively over all outgoing edges.
Mark all visited nodes.
In the end, answer ‘yes’ if node n is marked
Otherwise ‘no’.
Solution-algorithm for REACH
Idea for (search algorithm):
Start at node 1. Run recursively over all outgoing edges. Mark all visited nodes. In
the end, answer ‘yes’ if node n is marked, otherwise ‘no’.
S = {1}
mark the node 1
as long as S = {} :
select an i ∈ S and remove i from S
∀ edge (i, j) ∈ E, if j is unmarked:
add j to S and mark j
If node n is marked, answer ‘yes’,
otherwise ‘no’
Complexity of REACH
 Resource requirement: time requirement and
space/memory requirement
 Time requirement: each node is visited only
once.
 Space requirement: in the worst case, all n
nodes are marked.
 In the worst case, each edge must be
examined once (maximal n^2 , assuming no
multiple edges are allowed).
Complexity of REACH
•
In order to obtain an exact time specification
(approximately in seconds) for the time
configurations, one would have to assign a time
requirement to each calculation step.
•
For the sake of simplicity, a calculation step
corresponds exactly to one code line, for example:
add j to S and mark j
•
Actual time needed depends on implementation,
hardware, etc.
•
The same applies to space!
Estimation of complexity:
Landau (Big O) Notation
We use Landau Notation to get an abstract concept
of complexity independent of the implementation.
 Formal tool for describing the asymptotic behavior
of functions
 Relation between a function f(n) and a reference
function g(n)
 “f(n) behaves similarly to g(n)”
Complexity of REACH
• Resource Requirement: as a function of the input
variable (here, for example, the number of nodes n).
• Time requirement T(n) and space requirement S(n).
• Time requirement: each node is visited a maximum of
once. In the worst case, each edge must be examined
once (maximal n^2 edges).
•
⇒ T(n) = O(n^2)
• Space requirement: In the worst case, all n nodes are
marked.
•
⇒ S(n) = O(|V|)
Summary: REACH
No exact representation of G.
Assumption: random access to E.
How is the node i selected from S ?
Time requirement: O(n^2 ).
Space requirement : O(n).
Can be solved by search algorithm.
Summary
 Everyone has an intuitive notion of the difficulty of
problems
 Complexity theory: formal approach to complexity
 Landau Notation: asymptotic constraints for resource
requirements
 Concrete example of a problem: graph reachability
(REACH)
 REACH can be solved in O(n^2 ) time and O(n) space.
 Is this efficient?