OrderMergeDedup:
Efficient,Failure-ConsistentDeduplicationonFlash
ZhuanChenandKaiShen(UniversityofRochester)
I/ODeduplicationFailure-Consistency
EfficientFailure-Consistency
Softupdates-styleI/Oorderingforfailure-consistency
-  Afailurecanonlyleavegarbage(canbereclaimedlater)
Example:newwrite(dup.content)
Lower-than-actualref.
Incr.Pref. (prematureblockdeleHon)
Order1: MapL2P
✗
✓Order2: Incr.Pref.
MapL2P
Higher-than-actualref.(garbage)
✗
(CompletesoQupdates-stylewriteorderingindifferentwritecondiHons)
Cyclicdependencies
-  PreventmetadataI/Omerging&complicateimplementation(makesoftupdatescostlyforfilesystems[Seltzeretal.2000])
-  Adeduplicationsystemhasrelativelysimplesemantics(comparedtoafilesystem)
-  We(1)carefullydesignallI/Opaths;(2)delaynon-criticalI/O(thattheclientcompletionsignaldoesnotdependon)
Example:(1)Overwrite(dup.content)
MapL2P
Example:(2)Newwrite(dup.content)
(1) Incr.P1ref.
.
p
e
d
Decr.P2ref.
c
i
l
c
y
c
h
t
(1) Incr.P1ref.
MapL2P
Wi
(2)
Incr.P3ref.
MapL2P
Decr.P2ref.
Metadatablock
Metadatablock
Metadatablock
1 Metadatablock2
1
2
(2) Incr.P3ref.
MapL2P
MapL2P
CompleHontotheclient
(1) Incr.P1ref.
Metadatablock1 Metadatablock2
Delay Decr.P2ref.
AnticipatoryI/Odelayandmerging
(2) Incr.P3ref.
MapL2P
-  ShortI/Odelayinanticipationfornear-futuremerging
Metadatablock1 Metadatablock2
-  ReminiscentofI/Oanticipationscheduling[Iyeretal.2001]
I/OdeduplicationhelpsreduceFlashwear
…
…
L2Pmapping
P1 Write:ABC(ref.=2)
Write:ABC L1
…
Write:ABC L2
…
…
Logicalblocks
Physicalblocks(onFlash)
Metadatamaintenance
-  (1)Logical-physicalblockmapping
(2)Physicalblockfingerprints
(3)Physicalblockreferencecounts
-  Needtomaintainfailure-consistencyofdata/metadata
Existingapproachesforfailure-consistency
-  Relyonnon-volatileRAMorsupercapacitors/batteries
[Srinivasanetal.2012;Chenetal.2011;Guptaetal.2011]
-  Checking/repairtools[Quinlanetal.2002]
-  Redologging(extraloggingwrites)[Meisteretal.2010]
-  Shadowing(extraindexblockwrites)[Tarasovetal.2014]
Challenge:metadata&failure-consistency-inducedI/O
shouldn’tsignificantlydiminishthededuplicationI/Osaving
Evaluation
Original
Dmdedup
Failure−consistent write ordering
(A) Weak persistence model
+ Anticipatory I/O delay/merging
(B) Strong persistence model
Deduplicated physical block writes
Normalized I/O volume
Normalized I/O volume
Deduplicated physical block writes
1
0.8
0.6
0.4
0.2
0
Pa
ck
ag
Pa
BB
ck
a
ei
nd
ex
ge
en
up
ins
da
tal
l
te
ch
w
Ve
h
eb
icle
bro
co
Ha
do
un
ti
ws
ing
op
ng
YC
SB
/M
o
12×
12×
5×
10×
11×
11×
Pa
Pa
ck
BB
Ve
Ha
YC
SB
2
1.5
1
0.5
0
ck
ng
oD
B
ag
ei
nd
ag
ei
ex
up
d
ns
ate
en
c
tal
l
hw
hic
eb
le
co
bro
ws
un
do
op
ing
tin
g
/M
on
go
D
B
EvaluationSetup
-  ImplementedasacustomdevicemappertargetofLinux3.14.29
-  Platforms:Atom-basedtabletandXeon-basedservermachine
-  Mobileandserverworkloadswith23-73%writecontentduplication
Weakpersistencemodel
-  AwriteisreturnedearlywhilecorrespondingphysicalI/Ocanbedelayedtothenextflush
-  Wesave18-63%I/Owrites
Strongpersistencemodel
-  Awriteisreturnedbythededup.layerafterallcorrespondingphysicalI/Oarereturned
-  Wesave15-51%I/Owrites
-  AnticipatoryI/Odelay/mergingisparticularlyeffective