AMemoryModelforRISC-V
Sizhuo Zhang,
Muralidaran Vijayaraghavan,
Arvind
RISC-VWorkshop,November29,2016
WhynotSC/TSO?
Theybothhavesimplespecifications,both
axiomaticallyandoperationally
Butsimpleimplementationshavelow
performance
n
n
Strictorderingrequirementsformemory
instructions
Toimproveperformance,onemustmonitor
coherenceinvalidationtraffictopotentially
squashexecutedloads
WhynotPOWER/ARM?
Theiroperationalmodelsexposetoomuch
microarchitectural details
n
Branchspeculation,OOOexecution,rollbacketc
areexposedinthememorymodelspecification!
Theiraxiomaticmodelsaretoocomplexwith
nowell-understoodrelationto
microarchitecture
n
Onecannotsaywithconfidenceifaparticular
microarchitectural implementationobeysthe
model
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
MEMBAR
Branchr1!=1goto exit
Stb=1
Stc=1
Ld r2=c
r3=a+r2- 1
Ld r4=[r3]
exit:
Initiallyeverything’s0
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
MEMBAR
Branchr1!=1goto exit
Stb=1
Stc=1
Ld r2=c
r3=a+r2- 1
Ld r4=[r3]
exit:
Initiallyeverything’s0
(1)
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
(1)
MEMBAR
Branchr1!=1goto exit
(1)
Stb=1
Stc=1
Ld r2=c
r3=a+r2- 1
Ld r4=[r3]
exit:
Initiallyeverything’s0
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
(1)
MEMBAR
Branchr1!=1goto exit
(1)
Stb=1
Stc=1
Ld r2=c
r3=a+r2- 1
Ld r4=[r3]
exit:
Initiallyeverything’s0
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
(1)
MEMBAR
Branchr1!=1goto exit
(1)
Stb=1
Stc=1
Ld r2=c
r3=a+r2- 1
Ld r4=[r3]
exit:
Initiallyeverything’s0
(1)
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
(1)
MEMBAR
Branchr1!=1goto exit
(1)
Stb=1
Stc=1
Ld r2=c
(1)
r3=a+r2- 1
(a)
Ld r4=[r3]
exit:
Initiallyeverything’s0
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
(1)
MEMBAR
Branchr1!=1goto exit
(1)
Stb=1
Stc=1
Ld r2=c
(1)
r3=a+r2- 1
(a)
Ld r4=[r3]
(1)
exit:
Initiallyeverything’s0
Propertiesforanewmemorymodel
Simplespecificationwithout
microarchitectural detailslikeBranch
speculation,OOOexecution,rollback,etc
Butestablishcorrespondenceto
microarchitectureimplementations
WeakerthanSC/TSOforhighperformant,
simpleimplementations
InclusionofsufficientfencestoforceSC-like
behaviorwhennecessary
OurproposalforRISC-Vmemorymodel:
WMM
SimpleoperationalspecificationlikeSC,TSO,PSO
Instantaneous
Inorder
Execution
Processor
…
Processor
InstantaneousMemory
SC:
• Storesupdatememoryinstantly
• Loadreadsmemoryinstantly
OurproposalforRISC-Vmemorymodel:
WMM
SimpleoperationalspecificationlikeSC,TSO,PSO
Instantaneous
Inorder
Execution
Processor
Store
Buffer
…
Processor
Store
Buffer
InstantaneousMemory
TSO:
• Storesaredequeued inorder
• Whenastoreisdequeued fromstorebuffer,itupdatesmemory
instantly
• Loadreadstheyoungeststorefromstorebuffer,or(ifnotpresent)
memoryinstantly
OurproposalforRISC-Vmemorymodel:
WMM
SimpleoperationalspecificationlikeSC,TSO,PSO
Instantaneous
Inorder
Execution
Processor
Store
Buffer
…
Processor
Store
Buffer
InstantaneousMemory
PSO:
• Storesaredequeued inorderonlyforsameaddress
• Whenastoreisdequeued fromstorebuffer,itupdatesmemory
instantly
• Loadreadstheyoungeststorefromstorebuffer,or(ifnotpresent)
memoryinstantly
OurproposalforRISC-Vmemorymodel:
WMM
SimpleoperationalspecificationlikeSC,TSO,PSO
Instantaneous
Inorder
Execution
Processor
Store
Buffer
…
Invalidation
Buffer
Processor
Store
Buffer
Invalidation
Buffer
InstantaneousMemory
WMM:
• Storesaredequeued inorderonlyforsameaddress
• Whenastoreisdequeued fromstorebuffer,itupdatesmemory
instantly,removesaddressfromowninvalidationbufferandenters
everyotherinvalidationbufferinstantly
• Loadreadstheyoungeststorefromstorebuffer,or(ifnotpresent)
oldestentryininvalidationbuffer,or(ifnotpresent)memoryinstantly
• Oldestinvalidationbufferentrycanbethrownoutanytime
FencesinWMM
Acquire/ReconcileFence:ClearsInvalidation
buffer
Release/CommitFence:WaitsforStore
buffertobeflushed(non-atomically)
AxiomaticDefinitionofWMM
Memoryorderreorderingaxiom:
Can Reorder?
First
Second
Ld b
Stbv’
Acq/Reconcile Rel/Commit
Ld a
a!=b
No
No
No
Stav
Yes
a!=b
Yes
No
Acq/Reconcile
No
No
No
No
Rel/Commit
Yes
No
No
No
Loadreadstheyounger(inmemoryorder)of
n
n
LateststoreinmemoryorderforthataddressOR
Lateststoreinprogramorder(inthatthread)forthataddress
St-StFence:Commit
Ld-Ld Fence:Reconcile
St-Ld Fence:Commit+Reconcile
Ld-StFence:Notneeded
ImplementingWMM
FormallyProven:
OOO+ Single-threaded-correctness+ In-order-commit
+ ValuePrediction+ GlobalStoreAtomicity=WMM
Anexecutedloadwontgetsquashedlateras
longasitdoesn’tovertakeareconcileormemory
instructiontosameaddress
n
n
Nomonitoringofcoherenceinvalidations
Loadaddressspeculationallowed– squashedonlyif
predictedaddressiswrong
Allinstructionsarecommittedinorder
n
n
Storescannotovertakeloads
Prevents“out-of-thin-air”generationofvalues
ImplementingWMM
FormallyProven:
OOO+ Single-threaded-correctness+ In-order-commit
+ ValuePrediction+ GlobalStoreAtomicity=WMM
“Theoretically, the definition of the aq and rl bits allows for implementations
without global store atomicity. When both aq and rl bits are set, however,
we require full sequential consistency for the atomic operation which
implies global store atomicity in addition to both acquire and release
semantics. In practice, hardware systems are usually implemented with
global store atomicity, embodied in local processor ordering rules together
with single-writer cache coherence protocols.”
Writeback coherentcachehierarchytypicallysatisfiesGlobalStore
Atomicity
IfL1iswrite-through,easytoensureGlobalStoreAtomicityunlessthe
coreisSMT
n
SMTcoreswithL1write-throughcachesimplementa“non-multicopy-atomic”
memory
Don’tdoit
MappingC++11toWMM
C++11
WMM
Non-atomic Load
Load
Load Relaxed
Load
LoadConsume
Load;Acquire/Reconcile
LoadAcquire
Load;Acquire/Reconcile
LoadSC
Rel/Commit;Acq/Reconcile;Load;Acq/Reconcile
Non-atomicStore
Store
StoreRelaxed
Store
StoreRelease
Release/Commit;Store
StoreSC
Release/Commit;Store
UsingoperationalspecificationofWMMmakes
itstraightforwardtoderive/verifythismapping
Conclusion
WMMisamemorymodelwithsimple
specificationandpotentiallyhighperformant
implementations
n
BlendswellwithRISC-Vphilosophyandshould
beusedasthememorymodelforRISC-V
Advertisement:FormallyverifiedRISCV(subsetofRV32I)multicore
implementationinKami,ahardwareformalverificationplatform
Thankyou! [email protected]
[email protected]
[email protected]
Backup
WhynotRMO?
RMO’sdependencyrequirementsaretoo
strict
Thread1
Thread2
Sta=1
Ld r1=b
(1)
MEMBAR
Branchr1!=1goto exit
(1)
Stb=1
Stc=1
Ld r2=c
(1)
r3=a+r2- 1
(a)
Ld r4=[r3]
(1)
exit:
Initiallyeverything’s0
WhynotReleaseConsistency?
Fencesarenotstrongenoughtogive
SequentialConsistency
Initially,everythingis0
Thread1
Thread2
Stval =1
Ld r1=val
Release
Thread3
(1) Ld r2=flag
(1)
Acquire
Stflag =r1 (1) Ld r3=val
(0)
Non-cumulativeFences
Out-of-thin-airissue
Thread1
Thread2
Ld r1=x
Ld r2=y
Sty=R1
Stx=42
Initiallyeverythingis0
Finallyx=y=r1=r2=42
Noprocessorcanproducevaluesoutofthinair
n
Butincompletesetofaxiomsseeminglyallowsthis
Insistingonin-ordercommitsandadvertisingstoresonlyafter
committootherthreads/processorstakescareofthisissue
“The AMOs were designed to implement the C11 and C++11 memory
models efficiently. Although the FENCE R, RW instruction suffices to
implement the acquire operation and FENCE RW, W suffices to implement
release, both imply additional unnecessary ordering as compared to AMOs
with the corresponding aq or rl bit set.”
LitmusTestsforWMM
TestSB
WMMallowsthebehavior
- Ld overtakesStandCommit
P1
P2
I1:Sta 1
I2:Commit
Reconcile
I3:r1 =Ld b
I4:Stb1
I5:Commit
Reconcile
I6:r2=Ld a
AddReconciletoforbidthis
WMMallows:r1=0,r2=0
<a,1>
P1
P2
Reg state
Reg state
Store
buffer
Inv
buffer
<b,0>
<b,1>
Monolithicmemory
27
Store
buffer
Inv
buffer
<a,0>
LitmusTestsforWMM
WMMallowsthebehavior
- Ld overtakesLd
- Nodependencyordering
- Canbecausedbyvalue
predictioninhardware
TestMP+data
P1
P2
I1:Sta1
I2:Commit
I3:St ba
I4:r1 =Ld b
Reconcile
I5: r2=Ld r1
AddReconciletoforbidthis
WMMallows:r1=a,r2=0
<b,a>
<a,1>
Out-of-thin-airisimpossible
becauseofI2E
P1
P2
Reg state
Reg state
Store
buffer
Inv
buffer
Store
buffer
Monolithicmemory
28
Inv
buffer
<a,0>
<b,0>
WMM-S
Processor𝑝𝑠[𝑖]
Regstate𝑠
…
Stbuffer𝑠𝑏
Inv buffer𝑖𝑏
…
Monolithicmemory𝑚
ThesameabstractmachinestructureasWMM
Modelnon-multi-copy-atomicstores
n
n
n
Makeastorefromprocessori visibletoprocessorj beforethestoreupdates
monolithicmemory
Makeacopyofthestorefromthesb ofprocessori, andinsertthecopyintothe
sb ofprocessorj
Eachstorehasauniquetag,copieshavethesametag
Dequeue astorefromsb tomonolithicmemory
n
n
Allcopiesaredequeued fromsb
Allcopieshavetobetheoldestoneforthestoreaddressintheirrespectivesb
Copyingofamustbeconstrainedforper-locationSC
n
n
n
Eachsb ordersstoresforacertainaddressasalist
Combiningallsuchlistsfromallsb togetherformsapartialcoherenceorder
(<"# )ofthestoretagsforthataddress
Aftercopying,partialcoherenceordermustbestillacyclic
29
Storecopyexample
(Primesarecopies)
Insertedlater(younger)
↕
Insertedearlier(older)
P1sb
C’:𝑡/
A:𝑡.
Currentpartialcoherenceorder
n
n
𝑡, <"# 𝑡- <"# 𝑡. and𝑡/ <"# 𝑡𝑡, and𝑡/ areunrelated
IfwecopyCintosb ofP1asC’
n
n
Createcycle:𝑡. <"# 𝑡/ <"# 𝑡- <"# 𝑡.
Shouldnotbeallowed
IfwecopyAintosb ofP2
n
Createcycle:𝑡. <"# 𝑡.
30
P2sb
P3sb
A’:𝑡.
B:𝑡D:𝑡,
B’:𝑡C:𝑡/
LitmusTestsforWMM-S
• AddCommitinP2toforbid
thisbehavior
TestWRC
P1
P2
P3
I1:Sta 1
I2:r1=Ld a
Commit
I3:Stb r1
I4:r2 =Ld b
I5:Reconcile
I6:r3=Ld a
• Reconcilepreventsloads
fromreadingstalevalues-acquire
WMM-Sallows:r1=1,r2=1,r3=0
P1sb
<a,1>
P2sb
P3sb
<a,1>
m
• Commitgloballyadvertises
observedstores-- release
<b,1>
31
LitmusTestsforWMM-S
TestIRIW
P1
P2
P3
P4
I1:Sta 1
I2:r1=Ld a
Commit
I3:Reconcile
I4:r2=Ld b
I5:Stb1
I6:r3=Ld b
Commit
I7:Reconcile
I8:r4=Ld a
WMM-Sallows:r1=1,r2=0,r3=1,r4=1
P1sb
<a,1>
P2sb
P3sb
<a,1>
<b,1>
m
32
P4sb
<b,1>
WMM-SImplementation
WMM-ScanbeimplementedusingOOO+non-atomicmemorysystem
n
e.g.memorysystemoftheARMFlowingModel(FM)[1]
n
WedonotneedstorebufferinOOO,becauseFMhasbuffers
OOOP1
OOOP2
OOOP3
OOOP4
ROB
ROB
ROB
ROB
Segment𝑠[1]
Segment𝑠[2]
Segment𝑠[3]
Segment𝑠[4]
Segment𝑠[5]
FM
Segment𝑠[6]
Monolithicmemorym
[1]Flur etal.“ModellingtheARMv8architecture,operationally:concurrencyandISA”,POPL2016
33
FM+OOO
P1
ROB
P2
Seg.𝑠[1]
Seg.𝑠[2]
Seg.𝑠[5]
Simplifiedversionof
FM(nofenceinFM)
ROB
P3
ROB
Seg.𝑠[3]
P4
ROB
Seg.𝑠[4]
Seg.𝑠[6]
Monolithicmemorym
Eachsegmentisabufferofmemoryrequests
n
n
n
KeepsFIFOorderingofrequeststothesameaddress
Flowrule:Theoldestrequestforsomeaddressinasegmentcanbemovedto
theparentsegmentormonolithicmemory
Bypassrule:Astorecanforwarditsdatatoaload,aslongasthereisnoother
requesttothesameaddressinbetween
OOOcommit
n
n
store:directlyinsertintosegment
Commitfence:ifanysegmentcontainsastoreobservedbythecommitsofthe
OOOprocessor,thenwecannotcommitthefence
Astoreobservedbycommitsof𝑃𝑖:eithercommittedby𝑃𝑖 or
returnedbyaloadcommittedby𝑃𝑖
34
CCM+OOO⊆ WMM
FM+OOO⊆ WMM-S
HowWMM/WMM-SsimulatesCCM/FM+OOO
n
n
WhenthemonolithicmemoryinCCM/FMisupdatedbyastore
w WMM/WMM-Sdequeues thatstorefrom𝑠𝑏tomonolithicmemory
WhenOOOcommitsaninstruction
w WMM/WMM-Sexecutesthatinstruction
WhenOOOPicommitsaloadLforaddressa withresultv
n
Considerwhereisv inCCM/FM+OOOwhenLcommits
n
v isinmonolithicmemoryofCCM
w
n
v hasbeenoverwrittenbyanotherstoreinmonolithicmemory
w WMMhaspreviouslyinserted<a,v>intoib ofps[i]
w
n
WMMexecutesLbyreadingmonolithicmemory
NowWMMcanexecuteLbyreadingib
visinstorebufferofOOOPi
w
w
IfvhasbeenobservedbycommitsofPibeforeLiscommitted,thenWMM/WMM-Scan
executeLbyreadinglocalsb
Otherwise,WMM-Sfirescopy<a,v>intolocalsb andletLreadit
35
ImpactofDisallowingLd-StReordering
Qualitativeanalysis
n
n
n
Storebuffercanalreadyhidethestoremisslatency
Storesarenotonthecriticalpathforsingle-threadperformance
Inextremecases,thespeculativestorequeuemaybefilledupwith
uncommittedstores
Quantitativeevaluation
n
n
n
n
Simulate8-coremultiprocessorusingESESCsimulator
RunSPLASH2xbenchmarks
CompareWMM,Alpha,andaggressiveimplementationsofSCand
TSO
Alpha=WMM+Ld-Streordering
w
Trytofindyoungerstorestocommitwhentheinstructionatthecommit
slotoftheROBcannotcommit
36
SimulationConfiguration
37
Results
AveragecyclestocommitstoresearlyinAlpha
NormalizedexecutiontimeanditsbreakdownatthecommitslotofROB
38
NormalizedexecutiontimeanditsbreakdownattheissueporttoROB
Non-AtomicMemory
Modelsfornon-atomicmemoryismorecomplicated
Weareunclearabouttheperformanceadvantageofnon-atomicmemory
n
n
n
Becauseourunderstandingofthemicroarchitectural sourcesfornon-atomic
memoryislimited
POWER:sharedwrite-throughL1duetoSMT
w OthersourcesinthehierarchystartingfromL2?
ARM:noclue
w Manylitmustestsfornon-atomicstoresarenotobservableonhardware
w WRC+addrs,WWC+addrs,IRIW+addrs (http://diy.inria.fr/cats/modelarm/all.html)
w WRC+addrs (http://www.cl.cam.ac.uk/~sf502/popl16/observations.pdf)
Onlybyunderstandingthemicroarchitectural reasonsfornon-atomic
memory,areweabletoanalyzethebenefitofit
39