The SHMI value for each trust is plotted against the trust’s expected number of deaths on ‘funnel plots’ as
displayed below. A funnel plot is a form of statistical process control chart. These kind of charts can be used to
identify unexpected variation in clinical outcomes. If the observed number of deaths falls outside of an
expected range the Trust in question will be considered to have a higher or lower SHMI than expected. The
range, the extremes of which are called control limits, can be calculated in a variety of ways and are shown on
the funnel plots. For the SHMI two options are presented: exact Poisson control limits at a 99.8% level and
over dispersion control limits at a 95% level.
The NHS IC has agreed with the Technical Advisory Group that the SHMI should be published with two
different bandings by which outliers are classified. This reflects different views within the group on the extent
to which the SHMI should be adjusted for “over dispersion” (excess variability in the results), and the
implications for the bandings used for publication purposes.
The reason we have done this is so that, as befits the “experimental” status of the publication, we can take
soundings from Trusts and other potential users of the SHMI as to the relative merits of such adjustment for
the purpose of future publication. We do realise that the publication of two sets of SHMI classifications may
be confusing, but we believe that collectively we will benefit in future from the opportunity for some shared
learning as a result of wider feedback on these approaches.
In practical terms this means that the values which have been adjusted for over dispersion will show fewer
Trusts with values outside of the expected range (“outliers”). These Trusts reporting as “outliers” will also
show larger differences between the “observed” and “expected” numbers of mortalities (compared with
those not adjusted for over dispersion).
There is a low risk that Trusts may be “false positives”, ie are reporting outside the expected range simply
because of normal fluctuations in their results. But there is also a higher risk of “false negatives” – where
Trusts appear within the expected range but who should be outside.
The SHMI values which are not adjusted for over dispersion will show a greater number of Trusts reporting as
outliers compared with the outputs where the calculations have been adjusted for over dispersion, and this
will include all of the Trusts reporting as outliers where there has been adjustment for over dispersion. These
will show a higher risk of identifying “false positives”, but a lower risk of “false negatives”.
The imperative for investigation is stronger in the case of the values which have been adjusted for over
dispersion, compared with those which are not adjusted.
Copyright © 2011, The Health and Social Care Information Centre. All Rights Reserved.
1
In the period from 1 April 2010 to 31 March 2011 the funnel plots below show there were:
14 trusts whose SHMI value was ‘higher than expected’ under both methods and;
14 trusts whose SHMI value was ‘lower than expected’ under both methods.
Copyright © 2011, The Health and Social Care Information Centre. All Rights Reserved.
Copyright © 2011, The Health and Social Care Information Centre. All Rights Reserved.
3