• Comment: My main concern is on the 20% error assigned to the W+jets
predictions that is questionable looking at the results of the section 7.3 It
seems to me that most of the fluctuations in data/MC ratios for the
different channels, shown in figs 17, 19 and 21, are due to statistical
fluctuations of the MC predictions which seem not to be included in the
error bars (correct?). If this is the case I suspect we are overestimating the
systematic effect on the W+jets predictions and adding this uncertainty to
the fit predictions has the effect to reintroduce the statistical fluctuations
in the high mass region. Furthermore (even if the overall effect is small)
for the e-mu channel this uncertainty makes no sense as the inverted
Delta(phi) control region is dominated by other sources and the W+jets
contribution is negligible.
A possible approach to evaluate the uncertainty in the high mass
region would be to estimate the sensitivity of the fit to the starting point
of the fitted region. Instead of tuning the starting point according to the
best chi2 you could choose 3 different starting point values and using the
differences in the fit result as an additional systematic uncertainty. I also
support the checks proposed by Monica and Jamie and in particular the
comparison of the partial chi2s in the high mass tail in order to
quantitatively motivate the choose of the fit function.
• Answer: In Fig 17, 19 and 21, the statistical error is included, but we
believe that the difference between data and MC is induced by not only
statistics but also systematics. At low mass region, the statistical error is
small; at high mass region we are always lack of statistics, it’s hard to tell
the difference is from statistics or systematics. By looking at the central
value in Fig 17,19 and 21, we decided to quote a conservative
systematics(20%) for all the mass region.
•
Besides, Fig 17, 19 and 21 are used to assign systematics for the total
backgrounds , not wjet; for Wjet shape uncertainty, we have an “enhanced
inverted dphi” control sample, Fig 25 & Fig 27. For emu channel, it’s hard
to construct such a control sample due to statistics reason, but we believe
that the shape uncertainty is introduced at generator level(the object
effect is covered by the scale/smear uncertainty items), and it’s channel
independent, so we also applied the shape uncertainty got from other
channels to emu channel.
•
In summary, we thought the 20% systematics is necessary
2. We tried different fit function, and the exp fit can give better chi^2/ndof ;
we understood this was not an ultimate test, given the point below. Furthermore, the direct comparison
of the chi2 didn't show a clearly better result from the double exponential fit.
Answer: We tried several functions that perform well, and quote the difference between them as
systematics
3. We can not decide our fit function only from Chi^2, since it's dominated by low mass region where
we have larger statistics. We are more interested with high mass region;
--> indeed, that is exactly the point of our request: for example, one can estimate, for the double fit
function, the chi^2 that is just calculated for bins above m_ll>1TeV (or whatever) from a fit that is
down to the full range. This could then be used to tell how well it is modeling the high mass region.
We would like to see a table showing the chi^2 for different m_ll ranges 200GeV+, 500 GeV+,1TeV+,
1.5TeV+, …and by looking at that could justify the fit function choice.
Answer: Effect of “fit choice” systematic is not large, and has little effect on the final limit plots.
•
•
•
•
4. single exp fit obviously under-estimates the background numbers, so we don't really want
to quote the difference between double exp fit and single exp fit as systematics ;
--> it would be extremely useful to document this also as form of ratios. Even at the
presentation you show the double fit function over the 300 geV UP range, but it is unclear
how the ratio behaves.
Therefore we would like to request plots which show the bkg fit with the uncertainty bands
around the ratio which clearly show the behavior at low and high mass and provides a
quantification of the agreement.
Answer: We have checked the ratio plots, and will include these plots in the next version of
note
eμ
e
eμ
μ
μ
e
•
•
•
•
5. Considering double exp fit can give very good background yield, so we decide to use
double exp fit as our fit function ;
--> again, in a paper you will need to justify this choice. Actually, discussing with Andreas
(Hoecker), he mentioned why one could not use the fit function usually employed for dijet
mass distributions, documented for instance in
http://arxiv.org/pdf/1407.1376v1.pdf
of course, it is not guaranteed to work well considering that your bkg composition is not
equivalent. Still, you understand that there is no guarantee that your fit works at high
mass unless you provide additional support to this choice.
Answer: Similar as comment 4, we tried this fit function (“Dijet” fit in slide #4), and quote an
additional systematics between different fit functions. The final effect to expect limit is very
small(slide #3)
•
•
•
6. For the validation, since the background mass shape is constructed with many kinds of
backgrounds, not a single source, so it's hard to find a template which can describe the mass
shape in different region. Finally, our bkgd fit is only used to cope with statistic of the original
background shape in high mass windows, if the yield of fit function can well describes the
original background shape, that's enough. And the fit error will cover the uncertainty.
--> the ratio plots requested above would allow to visualize the latter statement.
Concerning the usage of the CR, we would be happy to see what happens fitting the bkg
with a double or single fit function in the inverted DPhi region. Even if the fit is different, it
could support the statement related to the choice of the fit range
Answer: We have checked the ratio plots, and also have a table comparing the difference
between fit function and original MC input in different region, and it seems the difference
can be covered by the uncertainty.
We also quote an additional systematics on the fit range as suggested by Lorenzo.
eμ
e
μ
Black: double exp fit
Red: single exp fit