UMass Lowell Computer Science 91.503 Graduate Analysis of Algorithms Prof. Karen Daniels Fall, 2006 Lecture 3 Wednesday, 9/20/06 Amortized Analysis Overview Amortize: “To pay off a debt, usually by periodic payments” [Websters] Amortized Analysis: “creative accounting” for operations can show average cost of an operation is small (when averaged over sequence of operations, not distribution of inputs) even though a single operation in the sequence is expensive no probability is involved (unlike average-case analysis) result must hold for any sequence of these operations guarantee holds in worst-case analysis method only; no effect on code operation Overview (continued) 3 ways to determine amortized cost of an operation that is part of a sequence of operations: Aggregate Method Accounting Method find upper bound T(n) on total cost of sequence of n operations amortized cost = average cost per operation = T(n)/n same for all the operations in the sequence amortized cost can differ across operations overcharge some operations early in sequence store overcharge as “prepaid credit” on specific data structure objects Potential Method amortized cost can differ across operations (as in accounting method) overcharge some operations early in sequence (as in accounting method) store overcharge as “potential energy” of data structure as a whole (unlike accounting method) Aggregate Method: Stack Operations Aggregate Method find upper bound T(n) on total cost of sequence of n operations amortized cost = average cost per operation = T(n)/n same for all the operations in the sequence Traditional Stack Operations PUSH(S,x) pushes object x onto stack S POP(S) pops top of stack S, returns popped object O(1) time: consider cost as 1 for our discussion Total actual cost of sequence of n PUSH/POP operations = n STACK-EMPTY(S) time in Q(1) Aggregate Method: Stack Operations (continued) New Stack Operation MULTIPOP(S,k) pops top k elements off stack S pops entire stack if it has < k items MULTIPOP(S,k) 1 while not STACK-EMPTY(S) and k = 0 2 do POP(S) 3 k k-1 MULTIPOP actual cost for stack containing s items: Use cost =1 for each POP Cost = min(s,k) Worst-case cost in O(s) in O(n) source: 91.503 textbook Cormen et al. Aggregate Method: Stack Operations (continued) Sequence of n PUSH, POP, MULTIPOP ops initially empty stack MULTIPOP worst-case O(n) O(n2) for sequence Aggregate method yields tighter upper bound Sequence of n operations has O(n) worst-case cost Each item can be popped at most once for each push # POP calls (including ones in MULTIPOP) <= #push calls <= n Average cost of an operation = O(n)/n = O(1) = amortized cost of each operation holds for PUSH, POP and MULTIPOP source: 91.503 textbook Cormen et al. Accounting Method Accounting Method amortized cost can differ across operations overcharge some operations early in sequence store overcharge as “prepaid credit” on specific data structure objects Let ci be actual cost of ith operation Let ĉi be amortized cost of ith operation (what we charge) n n Total amortized cost of sequence of operations must cˆi be ci i 1 upper bound on total actual cost of sequence i 1 n Total credit in data structure = n must be nonnegative for all n cˆ c i 1 i i 1 i source: 91.503 textbook Cormen et al. Accounting Method: Stack Operations Operation Actual Cost Assigned Amortized Cost PUSH 1 POP 1 MULTIPOP min(k,s) 2 0 0 Paying for a sequence using amortized cost: start with empty stack PUSH of an item always precedes POP, MULTIPOP pay for PUSH & store 1 unit of credit credit for each item pays for actual POP, MULTIPOP cost of that item credit never “goes negative” total amortized cost of any sequence of n ops is in O(n) amortized cost is upper bound on total actual cost source: 91.503 textbook Cormen et al. Potential Method Potential Method amortized cost can differ across operations (as in accounting method) overcharge some operations early in sequence (as in accounting method) store overcharge as “potential energy” of data structure as a whole (unlike accounting method) Let ci be actual cost of ith operation Let Di be data structure after applying ith operation Let F(Di ) be potential associated with Di Amortized cost of ith operation: cˆi ci F( Di ) F( Di 1 ) Total amortized cost of n operations: n n cˆ (c i 1 i i 1 i n terms telescope F ( Di ) F ( Di 1 )) ci F ( Dn ) F ( D0 ) i 1 Require: F( Dn ) F( D0 ) so total amortized cost is upper bound on total actual cost Since n might not be known in advance, guarantee “payment in advance” by requiring F( Di ) F( D0 ) source: 91.503 textbook Cormen et al. Potential Method: Stack Operations Potential function value = number of items in stack guarantees nonnegative potential after ith operation Amortized operation costs (assuming stack has s items) PUSH: potential difference= F( Di ) F( Di 1 ) ( s 1) s 1 amortized cost = cˆi ci F( Di ) F( Di 1 ) 1 1 2 MULTIPOP(S,k) pops k’=min(k,s) items off stack potential difference= F( Di ) F( Di 1 ) k ' amortized cost = cˆi ci F( Di ) F( Di 1 ) k 'k ' 0 POP amortized cost also = 0 Amortized cost O(1) total amortized cost of sequence of n operations in O(n) source: 91.503 textbook Cormen et al. Dynamic Tables: Overview Dynamic Table T: array of slots Ignore implementation choices: stack, heap, hash table... if too full, increase size & copy entries to T’ if too empty, decrease size & copy entries to T’ Analyze dynamic table insert/delete Actual expansion/contraction cost is large Show amortized cost of insert/delete in O(1) Load factor a(T) = num[T]/size[T] table: a(T) = 1 full table: a(T) = 1 empty (by convention) source: 91.503 textbook Cormen et al. Dynamic Tables: Table (Expansion Only) Load factor bounds (double size when T is full): Sequence of n inserts on initially empty table WHY? Worst-case cost of insert is in O(n) Worst-case cost of sequence of n inserts is in O(n2) LOOSE Double only when table is already full. “elementary” insertion source: 91.503 textbook Cormen et al. Dynamic Tables: Table Expansion (cont) Amortized Analysis Aggregate Method: count only elementary insertions ci= i if i-1 is exact power of 2 1 otherwise n lg n total cost of n inserts = ci n 2 j n 2n 3n Accounting Method: i 1 j 0 charge cost = 3 for each element inserted intuition for 3 each item pays for 3 elementary insertions inserting itself into current table expansion: moving itself expansion: moving another item that has already been moved source: 91.503 textbook Cormen et al. Dynamic Tables: Table Expansion (cont) Amortized Analysis source: 91.503 textbook Cormen et al. F(T ) 2num[T ] size[T ] Potential Method: Value of potential function F(T) 0 right after expansion builds to table size by time table is full always nonnegative, so sum of amortized costs of n inserts is upper bound on sum of actual costs Amortized cost of ith insert Fi = potential after ith operation Case 1: insert does not cause expansion cˆi ci Fi Fi 1 1 (2numi sizei ) (2numi 1 sizei 1 ) 3 Case 2: insert causes expansion cˆi ci Fi Fi 1 numi (2numi sizei ) (2numi 1 sizei 1 ) 3 use these: numi 1 sizei 1 numi numi 1 1 sizei 2sizei 1 Dynamic Tables: Table Expansion & Contraction count elementary insertions & deletions Load factor bounds: (double size when T is full) (halve size when T is ¼ full): DELETE pseudocode analogous to INSERT Amortized Analysis Potential Method: same as INSERT F(T ) 2num[T ] size[T ] if a (T ) 1 / 2 Value of potential function F(T) = 0 for empty table size[T ] / 2 num[T ] if a (T ) 1 / 2 0 right after expansion or contraction builds as a(T) increases to 1 or decreases to ¼ always nonnegative, so sum of amortized costs of n inserts is upper bound on sum of actual costs = 0 when a(T)=1/2 = num[T] when a(T)=1 = num[T] when a(T)=1/4 source: 91.503 textbook Cormen et al. Dynamic Tables: Table Expansion & Contraction Amortized Analysis Potential Method source: 91.503 textbook Cormen et al. Dynamic Tables: Table Expansion & Contraction Amortized Analysis source: 91.503 textbook Cormen et al. F(T ) 2num[T ] size[T ] if a (T ) 1 / 2 Potential Method size[T ] / 2 num[T ] if a (T ) 1 / 2 Analyze cost of sequence of n inserts and/or deletes Amortized cost of ith operation Case 1: INSERT Case 1a: ai-1 >= ½. By previous insert analysis: c ˆi 3 holds whether or not table expands Case 1b: ai-1 < ½ and ai < ½ no expansion cˆi ci Fi Fi 1 1 (( sizei / 2) numi ) (( sizei 1 / 2) numi 1 ) 0 Case 1c: ai-1 < ½ and ai >= ½ no expansion cˆi ci Fi Fi 1 1 (2numi sizei ) (( sizei 1 / 2) numi 1 ) 3 Dynamic Tables: Table Expansion & Contraction Amortized Analysis Potential Method Amortized cost of ith operation (continued) Case 2: DELETE Case 2a: ai-1 >= ½. Case 2b: ai-1 < ½ and ai < ½ no contraction cˆi ci Fi Fi 1 1 (sizei / 2 numi ) (sizei 1 / 2 numi 1 ) 2 Case 2c: ai-1 < ½ and ai < ½ contraction cˆi ci Fi Fi 1 (numi 1) (sizei / 2 numi ) ( sizei 1 / 2 numi 1 ) 1 Conclusion: amortized cost of each operation is bounded above by a constant, so time for sequence of n operations is O(n). source: 91.503 textbook Cormen et al. Example: Dynamic Closest Pair S S S source: “Fast hierarchical clustering and other applications of dynamic closest pairs,” David Eppstein, Journal of Experimental Algorithmics, Vol. 5, August 2000. Example: Dynamic Closest Pair (continued) S S S source: “Fast hierarchical clustering and other applications of dynamic closest pairs,” David Eppstein, Journal of Experimental Algorithmics, Vol. 5, August 2000.
© Copyright 2024 Paperzz
