K Delphes tuning in CheckMATE:
The present and the future
http://checkmate.hepforge.org
arXiv:1312.2591
M. Drees, H. K. Dreiner, J. S. Kim, D. Schmeier, J.Tattersall
Universität Heidelberg
K
K Contents
p Core Delphes changes
R Final state tunes
N Outstanding issues
N Non-detector issues
*I’ve made the assumption that the audience is familiar with Delphes
and detectors in general
**Please ask lots of questions otherwise this talk is far too short
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
2
Delphes
Core
Changes
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
3
K Delphes Improvements
Added ’flag’ functionality
p Many analyses require multiple ‘types’ of the same final state
object
R For example, a single lepton analysis may only require very loose
lepton id to veto di-lepton final states efficiently
N However, lepton is then required to fulfil much tighter conditions
to guard against fakes
B Also often seen with b-jets and taus
p Delphes did not have functionality to implement this consistently
R We require that a tighter object is always also a looser object
N E.g. → A tight electron will always also be a loose electron
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
4
K Delphes Improvements
Added ’flag’ functionality
p Big improvement is that we only run Delphes once irrespective of
the number of analyses
R Single run even with ATLAS and CMS analyses
p Minimal number of objects stored in Root output
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
5
Isolation
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
6
K Delphes Improvements
Isolation
p Flag functionality also allows for more isolation conditions
R Many analyses have isolations conditions based both on tracks
and calorimeter information
N Also both absolute and relative isolation conditions can be
applied
B Loose electron isolation settings often not stated
R Electron id requires an intrinsic isolation
N We phenomenologically use a ‘loosest’ isolation
B
Σ(calo(PT ))−`(PT )
`(PT )
< 0.2, within dR = 0.2
p Threshold energy of tracks/calo cells considered for isolation not
stated in analyses
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
7
K Delphes Improvements
Track isolation
p We use Σ(pT > 500) MeV (private communication)
R ATLAS-CONF-2014-032 now states Σ(pT > 400) MeV (2nd June)
Calorimeter isolation
p We use Σ(pT > 100) MeV (private communication)
entries
R Dominated by calorimeter noise below this
106
2012 data
s= 8 TeV
dT= 25 ns
105 <
µ>= 11.3
OF2
MF
104
103
ATLAS Preliminary
Tile calorimeter
102
-200 -150 -100 -50
0
50 100 150 200
Energy [MeV]
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
8
Final
State
Objects
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
9
K Delphes Improvements
Extensive changes made to Delphes tunes
p Majority of final state objects have been completely retuned
R
N
B
Q
Electrons
Muons
Taus
B-jets (charm-jets)
p Default Delphes
R Calorimeter → jets and MET
N Photons
B However, we have seen that these all require improvements
p All analyses use these common tunes as a basis
R If we see a particular analysis fails, we ‘fix’ within the analysis
p So far we have only implemented ATLAS tunings.
R CMS is on the todo list
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
10
K Delphes Improvements
Electrons
p We implement lookup tables based on PT and η
R Taken from arXiv:1110.3174 and ATL-COM-PHYS-2013-1287
(7 TeV)
N Implemented for loose, medium and tight electrons
B For each PT point we multiply by η distribution normalised to 1
1.00
CheckMATE
ATLAS
0.90
0.82
0.85
0.80
0.80
0.75
Average
0.78
0.76
0.74
0.70
0.65
0.600
CheckMATE
ATLAS
0.84
Efficiency
Efficiency
0.95
0.72
20
40
60
pT (GeV)
80
≥
80
0.70
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
2
1
0
Pseudorapidity
1
2
11
K Delphes Improvements
Electrons
p Now been updated with full 2-d binning @ 8 TeV
ATLAS-CONF-2014-032 (2nd June)
R Numbers are actually very similar
Efficiency
N As this is a conf note all numbers need to be read in by hand...
(any volunteers???)
1.3
1.2
1.1
ATLAS Preliminary
∫ L dt = 20.3 fb
-1
s = 8 TeV
1
ET: 60-80 GeV
Loose
LooseLH
Multilepton
Medium
MediumLH
Tight
VeryTightLH
0.9
0.8
0.7
0.6
-2 -1.5 -1 -0.5 0
0.5
1 1.5
2
η
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
12
K Delphes Improvements
Muons
p Same idea for muons (both segment tagged and combined)
R ∼ PT independent above 6 GeV
N Taken from ATLAS-CONF-2011-063 (7 TeV)
B ATLAS have now updated to 8 TeV but results are basically the
same
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
13
K Comparison
Z → ``
p Performance checks tell us, if our functions are correct (slightly
unfair on Delphes default.....)
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
14
K Delphes Improvements
Taus
p Signal and background efficiencies for both 1 and 3-prong taus
R Taken from ATLAS-CONF-2013-064 (8 TeV)
0.8
CheckMATE
ATLAS
Signal Efficiency
0.6
0.5
Tight
0.4
0.30
50
100
150
pT in GeV
200
CheckMATE
ATLAS
Background Efficiency
Loose
Medium
0.7
250
Loose
10-1
Medium
Tight
10-2 0
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
50
100
150
pT in GeV
200
250
15
K Delphes Improvements
B-tagging
p B-tagging implemented in ATLAS with an MVA
R For b-jets, c-jets and light-jets
N ATLAS-CONF-2012-040, ATLAS-CONF-2012-09,
ATLAS-CONF-2012-043 (all 7 TeV)
1.0
0.05
0.6
0.4
CheckMATE
Fit
ATLAS (Muon Data)
ATLAS (Top Data)
0.2
0.00
50
100
150
pT (GeV)
200
250
300
Light Jet Efficiency
Signal Efficiency
0.8
CheckMATE
Fit
ATLAS
0.04
0.03
0.02
0.01
0
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
100 200 300 400 500 600 700 800
pT in GeV
16
K Delphes Improvements
B-tagging
p We also parameterise the whole ROC curve
R Allows user to select any b-tagging efficiency and fake-rate is
automatically adjusted
102
ATLAS
CheckMATE
ATLAS
CheckMATE
104
c-Jet Rejection
Light Jet Rejection
105
103
102
101
101
1000.3
0.4
0.5
0.6
0.7
Signal Efficiency
0.8
0.9
1.0
1000.3
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
0.4
0.5
0.6
0.7
Signal Efficiency
0.8
0.9
1.0
17
K Delphes Improvements
B-tagging
p New results at 8 TeV need implementing
R ATLAS-CONF-2014-046, ATLAS-CONF-2014-004 (3rd July)
N Numbers have not changed much
B We correctly guessed deterioration at high PT
10
10
ATLAS Preliminary
tt simulation, s=8 TeV
p
1
jet
T
jet
> 20 GeV, | η | < 2.5
-1
b jets
c jets
light-flavour jets
Light-flavour jet rejection rate
Efficiency
Q Once again conf-notes....
10
4
ATLAS Preliminary
MV1
10
t t simulation,
3
jet
s=8 TeV
jet
p >20 GeV, | η |<2.5
T
10
2
10
-2
0
100
200
300
400
500
600
700
800
jet
p [GeV]
1
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
b-jet efficiency
T
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
18
K Delphes Improvements
B-tagging
p More important is the effect of other jets
R ATLAS-CONF-2014-014 (19th August)
N B-tagging gets worse if another jet lies close
B We need to experiment to see what works best
1.4
1.2
ATLAS Simulation Preliminary
Anti-k t R=0.4 Jets
25 GeV < jet p < 75 GeV
T
s= 8 TeV
75 GeV ≤ jet p < 200 GeV
MV1 70%
T
jet p ≥ 200 GeV
1
T
0.8
b-tagging efficiency
b-tagging efficiency
Q Once again conf-notes....
1.4
1.2
0.6
0.4
0.4
0.2
0.2
0.2
0.4
0.6
0.8
jet p ≥ 200 GeV
1
T
0.8
0.6
0
ATLAS Simulation Preliminary
Anti-k t R=0.4 Jets
25 GeV < jet p < 75 GeV
T
s= 8 TeV
75 GeV ≤ jet p < 200 GeV
T
MV1 70%
1
∆Rmin(q,b-jet)
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Track purity
19
K Delphes Improvements
B-tagging
p ∆R(jetq , jetb ) is difficult to be generator agnostic
R Implementing track impurities will also require work
b-tagging efficiency
N Centrality of b-quark should be easy
1.4
1.2
ATLAS Simulation Preliminary
Anti-k t R=0.4 Jets
25 GeV < jet p < 75 GeV
T
s= 8 TeV
75 GeV ≤ jet p < 200 GeV
MV1 70%
T
jet p ≥ 200 GeV
1
T
0.8
0.6
0.4
0.2
0
0.05
0.1
0.15
0.2
0.25
∆R(b-hadron,jet)
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
20
Outstanding
Issues
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
21
K Outstanding Issues
Calorimeter Smearing
p Delphes appears to undersmear jets (and thus MET) compared to
ATLAS
R Example in WW production
ATLAS WW (7 TeV)
ATLAS
Delphes
600
Events
500
400
300
200
100
00
20
40
60
T ) (GeV)
PT (``EMiss
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
80
100
22
K Outstanding Issues
Calorimeter Smearing
p Can also be seen in hadronic top reconstruction
R
N
B
Q
e.g stop search with one isolated lepton (arXiv:1407.0583)
t̃1 → t χ̃01 search targets one hadronic top (tN_med)
ATLAS → 130 < mjjj < 195 GeV
CheckMATE → 140 < mjjj < 185 GeV
p Does not effect ’normal’ jets + met searches so much
R Increased smearing on falling distribution has small effect on
acceptance
p Potentially serious for exclusive jet binning
R However... experiments should really be discouraged from
performing these cuts anyway
N Essetially a jet veto
B Not theoretically under control in electroweak production.
Q tt is far worse...
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
23
K Outstanding Issues
Trigger
p Ideally we would like to implement trigger independently of
analyses
R Lack of global trigger information makes this impossible
p We implement triggers individually for each analysis
R Unfortunately many analyses give no information about the trigger
p E.g. → atlas-conf-2013-035 (trilepton + MET) has no trigger
information
R Use single lepton trigger from atlas-conf-2012-104
N Use di-lepton trigger from atlas-conf-2012-049
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
24
K Outstanding Issues
Trigger
p No trigger turn-on curves given
R We believe this is the single biggest source of errors for lepton
analyses → especially for soft leptons
N Ideally, all triggers used should be documented centrally
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
25
Non-detector Simulation
Issues
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
26
K Non-detector Simulation Issues
Cutflows
p ATLAS now implements cutflows for all SUSY analyses
R A massive help when coding analyses
R We would like this to be extended to ALL analyses (including SM)
N CMS is severely lacking on this point......
B Only useful if reasonable statistics used
This can also benefit the experiments
p With the help of cutlows, CheckMATE alone has found...
R Wrongly documented PT cuts
N Wrongly weighted b-tag efficiencies
B Missing kinematical cuts
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
27
K Non-detector Simulation Issues
Fully documented Monte-Carlo settings
p Different Monte-Carlos can produce suprisingly different results
p Especially treatment of ISR
R Dumping all settings onto a wiki would be amazing
p SLHA files
R Monte-Carlo generator files
p Often generator level cuts are applied
p Is this at matrix element or full event level?
R Often non-standard and not transparent
p Easy to waste a lot of time trying to validate with a completely
different setup...
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
28
K Non-detector Simulation Issues
Fully documented errors
p We would like to combine signal regions and analyses to calculate
stronger limits
R Fully documented errors are a pre-requisite for this task
p Allows us to calculate error correlations
R Assuming uncorrelated errors only improves bounds marginally
N Assuming fully correlated errors significantly overestimates
bounds
p Each background must have every source detailed along with the
value
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
29
K Non-detector Simulation Issues
Stand-alone limit setting codes
p Even better would be stand-alone codes
R Produced by the experiment for each analysis
N Python code for CMS Rivet analysis developed but never released
p User is only required to enter expected events in each signal
region
R Code returns p-value
N Probably more chance of world peace tomorrow than this
happening for conf-notes in my lifetime...
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
30
A
Final
Plea
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
31
K A Final Plea
Conf note data
p This will almost certainly be ignored...
R But reading data from plots...
N
B
Q
K
Takes a huge amount of time
Is extremely boring
Induces errors
And is completely crazy
p We live in a digital age
R Can we please have plots in a digital format...
N please......
B please......
Q please......
Jamie Tattersall — Delphes tuning in CheckMATE: The present and the future
32