Recovery
03/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Integrity or correctness of data

Would like data to be “accurate” or “correct” at all times
EMP
Name
Age
White
52
Green 3421
Gray
1
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Integrity or consistency constraints


Predicates data must satisfy
Examples:
- x is key of relation R
- x  y holds in R
- Domain(x) = {Red, Blue, Green}
- a is valid index for attribute x of R
- no employee should make more than twice the average
salary
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Definition:


Consistent state: satisfies all constraints
Consistent DB: DB in consistent state
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Observation: DB cannot be consistent always!
Example: a1 + a2 +…. an = TOT (constraint)
Deposit $100 in a2: a2  a2 + 100
TOT  TOT + 100
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Example: a1 + a2 +…. an = TOT (constraint)
Deposit $100 in a2: a2  a2 + 100
TOT  TOT + 100
a2
TOT
13/30/2005
.
.
50
.
.
150
.
.
150
.
.
1000
.
.
1000
.
.
1100
Yan Huang - CSCI5330 Database
Implementation – Recovery
Transaction: collection of actions
that preserve consistency
Consistent DB
13/30/2005
T
Yan Huang - CSCI5330 Database
Implementation – Recovery
Consistent DB’
How can we prevent/fix violations?


cc: due to data sharing only
recovery: due to failures only
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Will not consider:



How to write correct transactions
How to write correct DBMS
Constraint checking & repair
That is, solutions studied here do not need to know constraints
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Storage hierarchy
x
Memory
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
x
Disk
Operations:


Input (x): block with x  memory
Output (x): block with x  disk
Read (x,t): do input(x) if necessary
t  value of x in block
Write (x,t): do input(x) if necessary
value of x in block  t
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Key problem Unfinished transaction
Example
13/30/2005
Constraint: A=B
T1: A  A  2
B  B2
Yan Huang - CSCI5330 Database
Implementation – Recovery
T1:
Read (A,t); t  t2
Write (A,t);
Read (B,t); t  t2
Write (B,t);
Output (A);
Output (B);
A: 8 16
B: 8 16
failure!
A: 8 16
B: 8
memory
13/30/2005
disk
Yan Huang - CSCI5330 Database
Implementation – Recovery
Undo logging
T1:
Read (A,t); t  t2
Write (A,t);
Read (B,t); t  t2
Write (B,t);
Output (A);
Output (B);
A:8 16
B:8 16
memory
13/30/2005
A=B
A:8 16
B:8 16
disk
Yan Huang - CSCI5330 Database
Implementation – Recovery
<T1, start>
<T1, A, 8>
<T1, B, 8>
<T1, commit>
log
One “complication”


Log is first written in memory
Not written to disk on every action
memory
A: 8 16
B: 8 16
Log:
<T1,start>
<T1, A, 8>
<T1, B, 8>
13/30/2005
A: 8 16
B: 8
Yan Huang - CSCI5330 Database
Implementation – Recovery
DB
Log
BAD STATE
#1
One “complication”

Log is first written in memory
Not written to disk on every action
memory
A: 8 16
B: 8 16
Log:
<T1,start>
<T1, A, 8>
<T1, B, 8>
<T1, commit>
13/30/2005
A: 8 16
B: 8
DB
...

<T1, B, 8>
<T1, commit>
Yan Huang - CSCI5330 Database
Implementation – Recovery
Log
BAD STATE
#2
Undo logging rules
(1) For every action generate undo log record
(containing old value)
(2) Before x is modified on disk, log records pertaining
to x must be on disk (write ahead logging: WAL)
(3) Before commit is flushed to log, all writes of
transaction must be reflected on disk
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Recovery rules:
Undo logging
(1) Let S = set of transactions with <Ti, start> in log, but no
<Ti, commit> (or <Ti, abort>) record in log
(2) For each <Ti, X, v> in log,
in reverse order (latest  earliest) do:
- if Ti  S then
- write (X, v)
- output (X)
(3) For each Ti  S do
- write <Ti, abort> to log
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
What if failure during recovery?
No problem!  Undo idempotent
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
To discuss:



Redo logging
Undo/redo logging, why both?
Checkpoints
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Redo logging (deferred modification)
T1:
Read(A,t); t t2; write (A,t);
Read(B,t); t t2; write (B,t);
Output(A); Output(B)
A: 8 16
B: 8 16
memory
13/30/2005
output
A: 8 16
B: 8
DB
Yan Huang - CSCI5330 Database
Implementation – Recovery
<T1, start>
<T1, A, 16>
<T1, B, 16>
<T1, commit>
LOG
Redo logging rules
(1) For every action, generate redo log record
(containing new value)
(2) Before anything is modified on disk (DB), all log
records for transaction that modify things (including
commit) must be on disk
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Recovery rules:
Redo logging
(1) Let S = set of transactions with
<Ti, commit> in log
(2) For each <Ti, X, v> in log, in forward
order (earliest  latest) do:
- if Ti  S then Write(X, v)
Output(X)
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Recovery is very, very
SLOW !
Redo log:
...
First
Record
(1 year ago)
13/30/2005
...
T1 wrote A,B
Committed a year ago
--> STILL, Need to redo after crash!!
Yan Huang - CSCI5330 Database
Implementation – Recovery
...
Last
Record
Crash
Undo Recovery – Better?

The first record without commit or abort should not be
too far away from crash
...
...
...
first record without commit or abort

But immediate update itself is expensive
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Solution: Checkpoint
(simple version)
Periodically:
(1) Do not accept new transactions
(2) Wait until all transactions finish
(3) Flush all log records to disk (log)
(4) Flush all buffers to disk (DB) (do not discard buffers)
(5) Write “checkpoint” record on disk (log)
(6) Resume transaction processing
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Example: what to do at recovery?
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
...
<T3,C,21>
...
<T2,commit>
...
<T2,B,17>
...
Checkpoint
...
<T1,commit>
...
<T1,A,16>
Redo log (disk):
Crash
Example: what to do at recovery?
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
...
<T3,C,21>
...
<T2,commit>
...
<T2,B,17>
...
Checkpoint
...
<T1,commit>
...
<T1,A,8>
Undo log (disk):
Crash
Nonquiescent Checkingpointing


“Quiescent” checkpointing stalls DB
“Nonquiescent” checkpointing admits new transactions
while checkpointing
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Nonquiescent Checkpoint Rules - undo


Write and flush a log record <START CKPT(T1, …,
Tk)> where T1,…,Tk are active transactions
When all T1, …, Tk have completed, write and flush a
log record <END CKPT>
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Example: what to do at recovery?
<T3,C,21>
<END CKPT>
...
<T1,commit>
...
<T3,B,17>
...
<T2,commit>
<CKPT(T1,T2)>
<T2,B,12>
...
<T1,A,8>
Undo log (disk):
Crash after end of checkpoint
Only need to undo all the incomplete
transactions started after <START CKPT>
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Crash
Example: what to do at recovery?
…
<T3,C,21>
...
<T1,commit>
...
<T3,B,17>
...
<T2,commit>
<CKPT(T1,T2)>
<T2,B,12>
...
<T1,A,8>
Undo log (disk):
Crash before end of checkpoint
Only need to undo all incomplete transactions
started after <START CKPT> and those in
<CKPT (…)>
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Crash
Nonquiescent Checkpoint Rules - redo



Write and flush a log record <START CKPT(T1, …, Tk)
where T1,…,Tk are active transactions
Flush all the updates of the transactions committed
before <START CKPT(T1,…,Tk)>
Write and flush a log record <END CKPT>
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Example: what to do at recovery?
<T3,C,21>
<END CKPT>
...
<T1,commit>
...
<T3,B,17>
...
<T2,commit>
<CKPT(T1,T2)>
<T2,B,12>
...
<T1,A,8>
Redo log (disk):
Crash
Crash after end of checkpoint
Only need to redo all the transactions committed after the
latest <START CKPT>
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Example: what to do at recovery?
…
<T3,C,21>
...
<T1,commit>
...
<T3,B,17>
...
<T2,commit>
<CKPT(T1,T2)>
<T2,B,12>
...
<T1,A,8>
Redo log (disk):
Crash before end of checkpoint
Only need to redo all the transactions
committed after the latest successful <START
13/30/2005
Yan Huang - CSCI5330 Database
CKPT>
Implementation – Recovery
Crash
Key drawbacks:


Undo logging: need to update immediately, increasing I/O
Redo logging: need to keep all modified blocks in memory until
commit
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Solution: undo/redo logging!
Update  <Ti, Xid, New X val, Old X val>
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Rules



Page X can be flushed before or after Ti commit
Log record flushed before corresponding updated
page (WAL)
Flush log at commit
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Non-quiesce checkpoint
...
for
undo
...
L
O
G
13/30/2005
Start-ckpt
active TR:
Ti,T2,...
...
end
ckpt
Flush
dirty
buffers
Yan Huang - CSCI5330 Database
Implementation – Recovery
...
Examples what to do at recovery time?
no T1 commit
L
O
G
...
T1,a
...
Ckpt
T1
...
Ckpt
end
 Undo T1 (undo a,b)
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
...
T 1b
Example
L
O
G
ckpt-s
T1
T1
Ckpt
T1
T1
...
... T1 ...
...
...
...
...
a
b
end
c
cmt
 Redo T1: (redo b,c)
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Examples what to do at recovery time?
no T1 commit
L
O
G
...
T1,a
...
Ckpt
T1
...
 Undo T1 (all the actions)
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
...
T 1b
Example
L
O
G
ckpt-s
T1
T1
...
... T1 ...
...
a
b
T1
T1
...
...
...
c
cmt
 Redo T1: (redo b,c, a and the actions after last
successful ckp-s)
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Recovery process:

Backwards pass (end of log -> latest checkpoint start)



Undo pending transactions


construct set S of committed transactions
undo actions of transactions not in S
follow undo chains for transactions in
(checkpoint active list) - S
Forward pass (latest successful checkpoint start -> end of log)

redo actions of S transactions
start
checkpoint
13/30/2005
backward pass
forward pass
Yan Huang - CSCI5330 Database
Implementation – Recovery
Recovery with Checkpoint Summary

Logging still obey the logging rules


Recovery still obey rules




undo, redo, undo-redo
Undo: undo all incomplete transactions
Redo: redo all completed transactions
Undo-redo: combined
Checkpoint provides a way to limit the transactions
needing undo or redo
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
WAL
Summary
undo
redo
Undo-redo
commit
commit means all
updates on disk
Commit means some
updates may start to
migrate to disk
Commit does not signal
disk update
recovery
Undo all incomplete
transactions
Redo all committed
transactions
Undo all incomplete
transactions and redo all
committed transactions
Nonquiescent
ckpt
<end ckpt(T)> means
every tran started before
the matching <start ckpt
(T)> is on disk
<end ckpt(T)> means
every tran committed
before the matching
<start ckpt(T)> is on disk
<end ckpt(T)> means
every update before the
matching <start ckpt(T)> is
on disk
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery
Summary


Consistency of data
One source of problems: failures
- WAL
13/30/2005
Yan Huang - CSCI5330 Database
Implementation – Recovery