The Ethics of Transparency: Publication of Cardiothoracic Surgical Outcomes in
the Lay Press
Jeffrey P. Jacobs, Robert J. Cerfolio and Robert M. Sade
Ann Thorac Surg 2009;87:679-686
DOI: 10.1016/j.athoracsur.2008.12.043
The online version of this article, along with updated information and services, is
located on the World Wide Web at:
http://ats.ctsnetjournals.org/cgi/content/full/87/3/679
The Annals of Thoracic Surgery is the official journal of The Society of Thoracic Surgeons and the
Southern Thoracic Surgical Association. Copyright © 2009 by The Society of Thoracic Surgeons.
Print ISSN: 0003-4975; eISSN: 1552-6259.
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
ETHICS IN CARDIOTHORACIC SURGERY
The Ethics of Transparency: Publication of
Cardiothoracic Surgical Outcomes in the Lay Press
Jeffrey P. Jacobs, MD, Robert J. Cerfolio, MD, and Robert M. Sade, MD
The Congenital Heart Institute of Florida, Division of Thoracic and Cardiovascular Surgery, All Children’s Hospital and Children’s
Hospital of Tampa, University of South Florida College of Medicine, Cardiac Surgical Associates of Florida, Saint Petersburg and
Tampa, Florida; Section of Thoracic Surgery at University of Alabama at Birmingham, Division of CardioThoracic Surgery,
Department of Surgery, Birmingham, Alabama; Division of Cardiothoracic Surgery, Department of Surgery, Institute of Human
Values in Health Care, Medical University of South Carolina, Charleston, South Carolina
C
ardiothoracic surgical (CTS) outcomes have been
published in the lay press for nearly 2 decades.
Pressures to expand such publication come from many
different areas and cannot be resisted indefinitely. Evidence exists that contemporary reporting of outcomes
data is based on flawed methodologies that potentially
mislead and deceive. Such deceptions may harm patients, surgeons, and hospitals in various ways, and could
undermine quality of surgical care and patients’ access to
it. Yet, public reporting of outcomes can also be beneficial to all concerned, but only if relevant data are accurate
and the formats in which they are reported are valid and
easily understood.
In this essay, we review the early history of public
reporting of CTS outcomes, discuss potentially negative
aspects of public reporting, and suggest solutions to
these problems. We then consider the positive aspects of
public reporting and provide recommendations for the
future. We conclude that CTS data should be collected
and analyzed under the direction of professional medical
societies and that reports of outcomes based on such data
should be published in the lay press.
where. These issues brought into question the quality of
the data that were being reported to the public [2].
Despite these flaws, public reporting of CTS outcome
data continues because of the belief that data on individual surgeons and hospitals are useful to several distinct
groups: individual patients for choosing hospitals and
surgeons, surgeons for improving their own outcomes
and hospitals for quality improvement, and governmental entities for developing health-related laws and regulations. Critical analysis of public reporting, however, has
undermined these beliefs.
Problems with Public Reporting and Potential
Solutions
Negative aspects of public reporting have been identified
or alleged and have been used to disparage public
reporting. They are related to complexity adjustment,
incomplete or inaccurate data, reduced access to CTS by
the sickest patients, statistics that are unsound because
they do not account for limited sample size and are based
on flawed administrative databases, and insufficiently
trained journalists.
The History of Public Reporting
Complexity Adjustment
In December 1990, New York state officials publicly
released hospital-specific data on raw as well as riskadjusted mortality for patients who underwent coronary
artery bypass grafting (CABG). In January 1992, specific
mortality figures for individual surgeons as well as for
hospitals were reported. In November 1992, Pennsylvania followed suit [1]. From the outset, the methodology of
New York’s Cardiac Surgery Reporting System (CSRS)
for adjusting risks and for comparing hospitals and
surgeons has been intensely criticized. The prognostic
accuracy of the CSRS model and whether it could adequately account for case–mix variations among surgeons
has been challenged. Anecdotal reports suggest that
some surgeons may have tried to avoid reporting adverse
statistics by referring some of their sickest patients else-
The data used in outcomes reports might not be of
suitable quality because they are not accurately adjusted
for the complexity of the cases and patients [3, 4].
Inaccurate public information can be misleading or deceptive and therefore dangerous. Unless the quality of
the data is high, public reporting of outcomes for cardiothoracic surgeons is not only useless, but might be
harmful to surgeons as well as patients. The harms will
remain until accurate risk-stratified data are publicly
available. A great deal of published information supports
this assertion [1].
No risk-adjusted model can take into account all of the
complexities of individual patients, according to
Dranove’s review of the New York experience in 2003:
Accepted for publication Dec 12, 2008.
It is essential for the analysts who create report cards to
adjust health outcomes for differences in patients characteristic (risk adjustment). . . . But analysts can adjust for
Presented at the Fifty-fourth Annual Meeting of The Southern Thoracic
Surgical Association, Bonita Springs, FL, Nov 7–10, 2007.
Address correspondence to Dr Sade, Department of Surgery, Medical
University of South Carolina, 96 Jonathan Lucas St, Ste 409, PO Box
250612, Charleston, SC 29425; e-mail: [email protected].
© 2009 by The Society of Thoracic Surgeons
Published by Elsevier Inc
Dr Jacobs discloses a financial relationship with
CardioAccess.
Ann Thorac Surg 2009;87:679 – 86 • 0003-4975/09/$36.00
doi:10.1016/j.athoracsur.2008.12.043
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
680
ETHICS IN CARDIOTHORACIC SURGERY
PUBLIC REPORTING OF OUTCOMES
JACOBS ET AL
only characteristics that they can observe . . . Because of
the complexity of patient care, providers are likely to have
better information on patient’s conditions than even the
most clinically detailed database. Thus, providers can
improve their ranking by selecting patients on the basis of
characteristics that are unobservable to the analyst but
predictive of good outcomes [5].
The Society of Thoracic Surgeons (STS) Database—
comprising General Thoracic, Adult Cardiac, and STS
Congenital Heart Surgery Databases—includes methodology to address this problem. It currently includes 1051
participating sites and 3202 participating surgeons. More
than 100 publications in professional journals and textbooks have come from the STS Database, and it has been
federally funded to study quality improvement [6].
The STS Database is based on several fundamental
principles that facilitate accurate comparison of outcomes and quality improvement of health care for patients undergoing cardiothoracic operations. These principles include standardized nomenclature to facilitate
meaningful analysis of outcomes, a minimum database
data set with precise and transparent definitions of data
fields, accurate and verified mechanisms of adjustment
for complexity of the patients, and data verification.
CTS outcomes analysis requires case–mix adjustment,
because case–mix varies among programs, and without
such adjustment, many surgeons and programs caring
for high-risk patients will be inappropriately praised
or criticized. Case–mix adjustment could eliminate
perverse incentives for surgeons to avoid caring for the
highest-risk patients—those who might need an operation the most and who might benefit the most from the
procedure.
Case–mix adjustment can be variously accomplished
by complexity stratification, as is currently done in the
STS Congenital Database, or with formal risk modeling
and risk-adjusted mortality, as is currently done in the
STS Congenital Heart Surgery Databases. Table 1 compares these useful and acceptable techniques of case–mix
adjustment [7, 8].
No risk-adjusted model can take into account all of the
complexities of individual patients. However, the risk
modeling techniques used in the STS Database correlate
well with reality, as demonstrated by several studies
[9 –12]. The tools for complexity adjustment described in
these studies clearly demonstrate that the STS Database
uses appropriate methods to adjust properly for
case–mix.
Data Verification
The data used in outcomes reports might not be of
suitable quality because they are not complete or accurate [3, 4, 13–16]. By comparison, data in the STS Database are verified for completeness and accuracy in two
ways: an intrinsic verification process designed to rectify
inconsistencies of data and missing elements of data, and
an on-site audit program that verifies data at its primary
source. Data in the STS Adult Cardiac Surgery Database
and the STS Congenital Heart Surgery Database are
randomly audited as part of a formal on-site audit pro-
Ann Thorac Surg
2009;87:679 – 86
Table 1. Two Methods of Case–Mix Adjustment (or RiskAdjustment) [7]
Risk-Adjusted Mortality Rate
Complexity Stratification
The mortality rate is adjusted
for differences in the
composition of the patient
population at the hospital
of interest and the
comparison group. It is an
estimate of what a given
hospital’s mortality rate
would be if its case–mix
were the same as the
comparison group.
Estimates a hypothetical
quantity
Usually requires statistical
modeling
Produces a single summary
measure
Requires assumptionsa
Used in the STS Adult
Cardiac Database
A method of analysis in which
the data are divided into
relatively homogeneous
groups (called strata). The
data are analyzed within
each stratum.
Compares actual observed
mortality rates
Simple, does not require a
model
Produces a separate summary
measure for each stratum
No assumptions
Used in the STS Congenital
Heart Surgery Database
a
Assumes that a hospital that does well with low-risk cases will do well
with high-risk cases.
STS ⫽ Society of Thoracic Surgeons.
gram conducted by an independent medical audit firm.
These data verification efforts have demonstrated that
the STS Database is a reliable source of complete and
accurate data [13].
Reduced Access to Health Care
Anecdotal reports have suggested that public reporting
can lead to referral of high-risk patients elsewhere,
creating an access problem for the sickest patients who
might benefit the most from CTS [17]. As already noted,
outcome data that lack accurate risk assessment and
adjustment are useless, but they may be worse than
useless—they may be dangerous. Providers caring for the
sickest patients are inappropriately penalized by the
absence of proper risk adjustment. Moreover, without
complexity adjustment, providers have reason to avoid
caring for high-risk, high-complexity patients, sending
them elsewhere instead. Severely ill patients are more
likely to travel to receive their care from centers of
excellence [17]. Apparently, severely ill patients believe
they have more to gain by traveling to busier, higherquality centers. This relocation allows lower-quality centers to avoid high-risk patients, thus improving their
apparent ranking. Similarly, Dranove showed that “there
has been a shift towards operating on healthier patients”
in lower-quality centers [17]. These lower-quality centers
are less likely to offer surgical intervention to high-risk
patients, even though these patients may benefit from it
the most.
According to Capps, the overall effect of the New York
report cards was to “change the incidence from sicker
patients toward healthier patents and lead to a higher
cost . . . and a deterioration of outcomes, especially
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
Ann Thorac Surg
2009;87:679 – 86
ETHICS IN CARDIOTHORACIC SURGERY
JACOBS ET AL
PUBLIC REPORTING OF OUTCOMES
Fig 1. The observed mortality rates and 95% confidence intervals
are shown for 175 surgeons. This graph plots the risk-adjusted 30day mortality rates after coronary artery bypass grafting in New
York between 1997 and 1999 and shows the ranked rates for individual surgeons with 95% confidence intervals. In the original publication of these data, the surgeons were named. The widths of the
confidence intervals in this figure reveal considerable uncertainty
about the true underlying mortality rates; however, this uncertainty
is not reflected in the rank given to a surgeon. This figure was reproduced with permission from the following publication [18]:
Spiegelhalter DJ. League tables. In: Armitage P, Colton T, eds. Encyclopaedia of Biostatistics. Chicester, U.K.: John Wiley and Sons;
2005:2478 –751.
681
dom variation. League tables are commonly used in
sports to rank teams or individual athletes by unadjusted
outcome data, usually wins and losses. League tables
always have winners and losers, with someone on the top
and someone on the bottom, even when no true difference exists between the subjects being ranked [18]. Figure 1 plots the risk-adjusted 30-day mortality rates after
CABG in New York between 1997 and 1999 and shows
the ranked rates for individual surgeons with 95% confidence intervals. In the original publication of these data,
the surgeons were named [19]. The widths of the confidence intervals in Figure 1 show few intervals that do not
overlap, revealing considerable uncertainty about the
true underlying mortality rates. This uncertainty, however, is not reflected in the ranks of specific surgeons [18].
Spiegelhalter has pointed out that if one thinks of the
intervals in Figure 1 as expressing probability distributions for the true mortality rates, and one then samples
those distributions and ranks each of the generated
samples, a set of plausible “true ranks” will be created.
As revealed in Figure 2, these ranks show substantial
uncertainty. The intervals for most surgeons are very
wide: only 2 of 175 can be confidently placed in the lowest
mortality quartile and only 6 in the highest mortality
quartile. Thus, “any ‘league table’ is largely spurious,
apart from possibly identifying some extreme cases that
can confidently be placed in, say, the top or bottom
quarter” [18].
among ill patients. We therefore conclude that the report
cards were welfare-reducing” [17]. One solution to this
potential problem of reduced access to health care is the
use of robust clinical data sets that adjust properly for
complexity of the patients so CTS outcomes can be
analyzed accurately.
Sample Size and Random Variation
The limited sample size of any individual surgeon’s or
hospital’s experience can lead to (1) wide fluctuations in
outcomes from year to year and (2) reporting of outcomes
as being different when they are actually statistically
similar and appear to be different due only to chance
rather than to true statistical differences. Without proper
adjustments for sample size and random variation, a shift
of one or two deaths a year can lead to a dramatic effect
on the reported outcomes of a surgeon or a hospital. This
effect has been clearly seen in both New York and
Pennsylvania, where small sample size and low mortality
rates led to wide swings in rankings from year to year,
despite absences of demonstrable differences in quality
of care. Failure to adjust properly for sample size and
random variation can mislead and deceive.
CTS outcomes have been publicly reported using
league tables, which are fundamentally flawed, unnecessary, and inappropriate. They use outcome data to rank
participants, with no adjustment for sample size or ran-
Fig 2. The median estimates and 95% intervals are shown for true
ranks of 175 New York surgeons. If one thinks of the intervals in
Fig 1 as expressing probability distributions for the true mortality
rates, and one then samples those distributions and ranks each of
the generated samples, a set of plausible “true ranks” for the surgeons is created, as shown in Fig 2. The true ranks in Fig 2 show
substantial uncertainty, with most surgeons having a very wide interval (only 2 out of 175 can be confidently placed in the “best”
quarter, and only 6 in the “worst” quarter). This figure was reproduced with permission from the following publication [18]:
Spiegelhalter DJ. League tables. In: Armitage P, Colton T, eds. Encyclopaedia of Biostatistics. Chicester, U.K.: John Wiley and Sons;
2005:2478 –751.
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
682
ETHICS IN CARDIOTHORACIC SURGERY
PUBLIC REPORTING OF OUTCOMES
JACOBS ET AL
Ann Thorac Surg
2009;87:679 – 86
Administrative Data and the Role of the Government
Fig 3. A funnel plot demonstrates risk-adjusted 30-day mortality
rates after coronary artery bypass grafting in New York between
1997 and 1999 for 175 New York surgeons. The solid horizontal line
shows the average 30-day mortality rates for these surgeons. The
two dotted lines show the 95% confidence intervals. The two dashed
lines show the 99.9% confidence intervals. Each dot represents a
surgeon. Only surgeons outside of the funnel are outliers. This figure
demonstrates that that the vast majority are not outliers. As
Spiegelhalter states, “The plot makes clear that there is no point in
carrying out a ranking exercise on those in the ‘funnel.’” This figure
was reproduced with permission from the following publication [18]:
Spiegelhalter DJ. League tables. In: Armitage P, Colton T, eds. Encyclopaedia of Biostatistics. Chicester, U.K.: John Wiley and Sons;
2005:2478 –751.
Spiegelhalter addressed the problem of small sample
size by describing the funnel plot, which plots mortality
rates of institutions or surgeons on a graph in conjunction with 95% and 99% binomial confidence intervals
centered around the average population mortality rate
[20]. A funnel plot is a mechanism to identify outliers in
performance without creating league tables. Figure 3
shows a funnel plot of the New York surgeons’ data and
demonstrates that the vast majority are not outliers.
Spiegelhalter states, “The plot makes clear that there is
no point in carrying out a ranking exercise on those in the
‘funnel’.”
Funnel plots are used to publicly report outcome data
in the United Kingdom Central Cardiac Audit Database
(CCAD) [21]. Since 2007, the STS Congenital Heart Surgery Database Report has used similar techniques that
allow identification of outliers without creating a league
table that ranks programs without true differences [22,
23]. The funnel plot explicitly demonstrates the substantial random sampling variation that occurs at low volumes and the difficulty in distinguishing among levels of
performance. As a consequence, surgeons and institutions with randomly high mortality rates are protected
from inappropriate conclusions about their data. Patients
and interested institutions are similarly protected from
making erroneous decisions in favor of surgeons and
institutions with randomly low mortality rates. This
methodology clearly demonstrates the availability of statistical tools to account for both small sample size and
random variation.
Government reports of CTS outcomes are based on
severely flawed administrative databases. Furthermore,
many nongovernmental organizations have advocated
public reporting of CTS outcomes based on these flawed
administrative data. The Consumers’ Checkbook Web
site (checkbook.org) uses administrative Medicare claims
data to provide information about the number of procedures a particular surgeon performs for a specific type of
operation. The Center for the Study of Services, which
owns this Web site, has successfully sued the United
States government to obtain Medicare claims data on all
physicians participating in the Medicare program [24]. At
this writing, the case is under appeal by the Department
of Health and Human Services and the Department of
Justice; chances for overturning the lower court’s verdict
are uncertain.
If successful again, the Consumers’ Checkbook organization would publish physician-level data from Medicare on its Web site. These ratings would reflect data in
the Medicare database, thus offering only a fraction of the
clinical experience of many surgeons. The Medicare
database is largely restricted to patients aged 65 years or
older, thus biasing the analysis and misleading those
who use it. Few patients who visit checkbook.org are
likely to be aware of this flaw or to understand its
meaning, despite the Web site’s attempt to explain it.
For example, consider a patient—let’s call him Joe Internet—who is searching for “the best” mitral valve surgeon to
repair his 44-year-old wife’s mitral valvar regurgitation. Joe
and his wife are interested in valve repair instead of
replacement. They visit the checkbook.org Web site and
view the ranking of surgeons who perform the most mitral
valve surgery in the United States. They are thrilled to tell
their friends that they have identified the best and busiest
surgeon. But have they? Or have they been misled?
Because many mitral valve repair operations are performed on patients aged younger than 65, a large proportion of the procedures are absent from this analysis.
The site has misled— or frankly deceived—Joe and his
wife about who does the most mitral valve repairs. In
addition to volume, checkbook.org also intends to list the
costs of a particular operation by a particular surgeon.
Yet, one of the few aspects of cost surgeons can partly
control is length of stay, but it is not listed. Inaccurate or
incomplete data are worse than no data at all.
Government agencies cannot fully understand the
complexities of medical and surgical outcomes analysis
without the leadership of professional medical and surgical societies. We must help those agencies understand
the best data sources and best reporting methodologies.
Accurate reporting of CTS outcomes requires reliance on
clinical databases rather than administrative databases.
A recent study compared data on isolated CABG results
from an audited and validated clinical registry with data
derived from a contemporaneous state administrative
database using the inclusion/exclusion criteria and risk
model of the Agency for Healthcare Research and Quality [25]. This study concluded, “Cardiac surgery report
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
Ann Thorac Surg
2009;87:679 – 86
ETHICS IN CARDIOTHORACIC SURGERY
JACOBS ET AL
PUBLIC REPORTING OF OUTCOMES
cards using administrative data are problematic compared with those derived from audited and validated
clinical data, primarily because of case misclassification
and nonstandardized end points.”
Three recent investigations compared coding of congenital cardiac disease from clinical databases with administrative databases that used the International Classification of
Diseases (ICD) coding system. They demonstrated that the
validity of coding congenitally malformed hearts using ICD
is likely to be poor [26 –28]. Several explanations of the poor
diagnostic accuracy of administrative databases that use
ICD codes are plausible: accidental miscoding, coding by
medical records clerks who have not seen the patient, contradictory or inaccurate information in the medical record,
lack of diagnostic specificity for cardiothoracic disease in
the ICD codes, and inadequately trained coding clerks.
Governmental administrative databases were created
to facilitate billing; they do not accurately reflect CTS
outcomes with the level of detail and accuracy necessary
for useful outcome analysis. Meaningful analysis of CTS
outcomes requires the use of clinical databases rather
than administrative databases.
The Role of the Lay Press
Lay press reports of CTS outcomes are problematic
because the average health care reporter does not have
the training and background to appreciate the science
behind the analysis of medical and surgical outcomes.
For example, on March 1, 2001, the Denver Post published
a front-page article, “Children’s Hospital Cardiology
Chief Told to Resign.” The reporter wrote, “The move
follows a report that shows the average mortality rate for
open-heart surgery at Children’s was 4.2%, or 30 children
out of 710 surgeries, between January 1997 and July 2000,
while the national average was 2.7%” [29]. A follow-up
Denver Post editorial the next day stated, “The hospital
also found that, through last June, 4.2 percent of the
children who had the procedure at Children’s Hospital
died. Nationally, the average mortality rate of such surgeries is 2.7 percent, according to the Society for Thoracic
Surgeons [sic]” [30]. The author of the front-page article,
a recent graduate of Colorado State University, failed to
consider four crucial points:
1. At that time, the STS Database had never published
an overall morality figure for congenital heart surgery.
2. If the STS Database had published an overall morality figure for congenital heart surgery at that
time, it would have been more than 4%.
3. Any mortality comparison must account for variations in case–mix.
4. Any mortality comparison must include confidence intervals based on sample size and random
variation.
The chair of the STS Congenital Heart Surgery Database at that time, Constantine Mavroudis, MD, wrote a
rebuttal to the Denver Post addressing the shortcomings
in their presentation of the data. The newspaper did not
publish his critique. This example raises the question:
683
Who is better prepared to present CTS outcomes, an
experienced cardiothoracic surgeon or a freshman reporter who recently finished college with an undergraduate degree in journalism? Publication of CTS outcomes
in the lay press requires substantial input from cardiothoracic surgeons who understand surgical complexities.
Positive Aspects of Public Reporting
Public reporting of CTS outcomes, if done properly, can
provide benefits to many stakeholders, including patients, surgeons, other health care professionals, hospitals, insurance companies, and the government. The
positive features of public reporting argue for transparency, understood as accurate and complete disclosure of
CTS outcomes.
Stakeholders Desire Transparency
As a profession, we must recognize that, like it or not, our
patients, insurance companies, news media, and several
levels of government want transparency. This desire
increased after recent high profile reports of medical
errors and related publications from the Institute of
Medicine. In 2000, it estimated that 3% to 4% of hospitalized patients experienced an adverse event, defined as
an injury caused by medical management. Slightly more
than half of these adverse events were attributed to
errors and were classified as preventable adverse events.
As many as 98,000 Americans die each year of medical
errors in hospitals [31]. This report generated substantial
interest in the evaluation of the quality of medical care.
In the United States, the idea of transparent disclosure
of the outcomes of medical treatment is exemplified by
the National Quality Forum [32]:
A not-for-profit membership organization created to develop and implement a national strategy for health care
quality measurement and reporting. . . . The mission of
the National Quality Forum is to improve the quality of
American healthcare by setting national priorities and
goals for performance improvement, endorsing national
consensus standards for measuring and publicly reporting on performance, and promoting the attainment of
national goals through education and outreach programs.
Clearly, transparency and public reporting is high
priority to the National Quality Forum.
Several insurance companies, including United Health
Care and Aetna, have created “data entry forms” for CTS
programs to document outcomes and submit the data to
the insurance companies, which use the outcomes reports to create “Centers of Excellence.” Similarly, several
state governments now track CTS outcomes.
The efforts of Pennsylvania and New York are well
known to the CTS community, as has been discussed.
Florida tracks pediatric cardiac surgical outcomes
through Florida Children’s Medical Services and its Children’s Cardiac Program. Similar efforts by the federal
government were foreshadowed in 2006 when the Centers for Medicare and Medicaid Services launched the
Physician Voluntary Reporting Program, which eventu-
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
684
ETHICS IN CARDIOTHORACIC SURGERY
PUBLIC REPORTING OF OUTCOMES
JACOBS ET AL
ally became the Physician Quality Reporting Initiative
and may eventually evolve into a “pay for performance”
program [33]. The desire of the federal government for
public reporting is further demonstrated by federal legislation proposed in 2007: S. 1693, the “Wired for Health
Care Quality Act” [34]. The bill provides “for the development of reports based on Federal health care data and
private data that is publicly available or is provided by
the entity making the request for the report in order
to . . . provide the public with reports on national, regional, and provider- and supplier-identifiable format.”
The STS has expressed this opinion regarding claims
data used to create outcomes reports (personal communication) [35]:
In our two decades of experience, it has become abundantly clear that claims (or billing) data are not sufficient
to measure outcomes and are incapable of allowing the
significant risk adjustment that can prevent patient access
disparities— usually for the sickest patients. Administrative data is rife with misclassification errors, which are
incredibly difficult to correct or account for in a reporting
system. Only a clinical database with sufficient clinical
variables can be risk adjusted enough to yield accurate
findings.
The widespread desire for transparent reporting of
CTS outcomes charges our profession with the responsibility to ensure that relevant data are provided in an
accurate and comprehensible format.
Public Reporting Can Lead to Quality Improvement
and Cost Reduction
The importance of public reporting goes far beyond the
desires of stakeholders; it can improve the quality of care
in cardiothoracic operations and reduce the cost of delivering care. Porter and Teisberg have argued that
transparency is essential to the redefinition of health care
based on value, which they define as health outcome
divided by cost [36, 37]. Public reporting is likely to
change referral patterns and individual surgeons’ numbers of procedures. More importantly, however, it may
lead to improvements in the overall quality of health care
by identifying the best providers so that we can learn
from them, and by identifying low performing centers
that can benefit from quality improvement initiatives.
Porter and Teisberg [36, 37] state, “Value in health care
is the health outcome per dollar of cost expended. If all
system participants have to compete on value, value will
improve dramatically.” They advocate measuring riskadjusted outcomes and costs. They also state, “If physicians fail to lead these changes, they will inevitably face
ever-increasing administrative control of medicine” [36, 37].
Professional Responsibility
Public reporting promotes surgical professionalism by
facilitating self-regulation and improving the state of our
surgical art [38 – 44]. The STS Database was created for
the purpose of improving the quality of care for CTS
patients. As a matter of professional obligation, the STS
created an excellent tool for reporting CTS outcomes. The
Ann Thorac Surg
2009;87:679 – 86
STS Database allows for the reporting of CTS outcomes
based on verified data that is adjusted and stratified for
operative complexity. It also allows for the use of composite measures to assess quality based on the combination of performance and compliance with structure, process, and outcomes indicators [45, 46]. The methodologies
used by the STS Database can be used to report CTS
outcomes accurately and in language that patients and
other laypersons can understand.
Implications, Inferences, and Conclusions
Reporting of CTS outcomes is already here, and expansion to encompass all of surgery seems inevitable. Insurance companies, governmental agencies, the news media, and others will attempt to report outcomes
independently if we do not collaborate with them. Medical and surgical professional societies are best suited to
analyze the outcomes of medical and surgical treatments,
and to implement strategies to improve these outcomes.
As a profession, we are ethically committed to serve the
best interests of patients and to optimize access to appropriate health care. Therefore, it is our responsibility to
lead the public reporting initiative. If we do not lead this
initiative, someone less qualified will.
Serving the medical interests of our patients entails
providing or otherwise making available information that
would be useful to them in deciding to whom they wish
to entrust their surgical care. This idea is a corollary of
informed consent for specific procedures: patients must
be provided sufficient information to accept or reject a
proposed treatment and, by extension, should be provided sufficient information to make decisions about
their care, including relevant surgical outcome data, by
surgeon and by institution. It follows, therefore, that
surgeons should support public reporting of outcome
data.
Mere reporting is not sufficient, however. Our obligation to serve the medical interests of our patients includes preventing avoidable harm. Current publicly
available surgical outcome information, mainly from administrative databases, is misleading at best. Deceptive
information of this kind threatens our patients by leading
them to make misinformed choices. Publication of inaccurate negative outcome data also threatens unjustifiably
to stigmatize individual surgeons and their institutions.
Another potential harm to patients is the possibility
that some physicians may feel a need to protect themselves from adverse public reporting of their own results
by denying interventions to patients with higher-thanaverage risks. Such a tactic would diminish some patients’ access to the appropriate cardiothoracic operation,
so is ethically unacceptable. Public reporting based on
accurate and reliable risk stratification would soften or
eliminate this disincentive to provide appropriate care.
A potentially important barrier to the use of the STS
Database for public reporting is that it was not created
with public reporting in mind. The current widespread
(although not universal) participation by cardiothoracic
surgeons nationally may be largely due to the promise to
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
Ann Thorac Surg
2009;87:679 – 86
ETHICS IN CARDIOTHORACIC SURGERY
JACOBS ET AL
PUBLIC REPORTING OF OUTCOMES
individual surgeons and surgical programs that their
data are confidential. The only publicly available reports
are combined national data; surgeon-specific and program-specific information can be reported only by or
with the approval of the surgeon or program in question.
Dependence on voluntary data submission by surgeons brings with it many difficulties, the greatest of
which is the likelihood that many might drop out of the
STS Database if surgeon-specific or program-specific
analyses based on its contents were to be publicly reported. It may be that the best way STS methodology
could be used for public reporting is if reporting of CTS
results were required by law and reporting agencies used
STS methodology. This strategy would maintain the
voluntary and confidential nature of the STS Database,
but would mandate that the data submitted for public
reporting be based on the same methodologies as those
used in the STS Database.
The clinical detail required for the STS Database is not
necessary for reimbursement purposes, but if such detail
were required for reimbursement, errors and mishandling of data would almost certainly burgeon. The completeness and accuracy of information currently reported
to the STS Database requires constant attention by those
who submit the information and those who manage the
database, including routine audits of submitted data by
an independent third party. Clinical data reported for
reimbursement purposes can be flawed because of incentives to maximize income by reporting erroneous or
misleading information [47]. A potential consequence of
mandatory reporting, therefore, is an unintended decrease in accuracy of the reported data. Data verification
efforts will need to be increased and adequately funded.
In the final analysis, we as a profession should participate in solving the daunting problems associated with
public reporting of CTS outcomes data, but not primarily
because of its inevitability, or the threat that someone
else will do it if we do not, or to protect ourselves from
unjustified stigmatization. We should participate, as individuals and as professional societies, because doing so
is required by our most fundamental professional commitment: to do what is best for our patients.
References
1. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG.
Systematic review: the evidence that publishing patient care
performance data improves quality of care. Ann Intern Med
2008;148:111–23.
2. Green J, Wintfeld N. Report cards on cardiac surgeons.
Assessing New York State’s approach. N Engl J Med 1995;
332:1229 –32.
3. Lacour-Gayet FG, Clarke D, Jacobs JP, et al. The Aristotle
score: a complexity-adjusted method to evaluate surgical
results. Eur J Cardiothorac Surg 2004;25:911–24.
4. Jacobs JP, Lacour-Gayet FG, Jacobs ML, et al. Initial application in the STS congenital database of complexity adjustment to evaluate surgical case mix and results. Ann Thorac
Surg 2005;79:1635– 49; discussion 1635– 49.
5. Dranove D, Kessler D, McClellan M. Is more information
better? The effects of “ report cards” on health J Polit Econ
2003;111:555– 88.
685
6. Society of Thoracic Surgeons. Welcome to the STS National
Database. Available at: http://www.sts.org/sections/
stsnationaldatabase/. Accessed April 14, 2008.
7. O’Brien SM. Complexity stratification vs. risk adjustment:
what is the difference? Presented at: The Society of Thoracic
Surgeons Advances in Quality & Outcomes Conference:
Minneapolis, MN; Nov 1–3, 2007.
8. O’ Brien SM. Z-Scores and confidence intervals. Presented
at: The Society of Thoracic Surgeons Advances in Quality &
Outcomes Conference. Minneapolis, MN; Nov 1–3, 2007.
9. Jacobs ML, Jacobs JP, Jenkins KJ, Gauvreau K, Clarke DR,
Lacour-Gayet FG. Stratification of complexity: the Risk Adjustment for Congenital Heart Surgery-1 method and the
Aristotle Complexity Score–past, present, and future. Cardiol Young 2008;18(suppl 2):163– 8.
10. Shroyer AL, Coombs LP, Peterson ED, et al. The Society of
Thoracic Surgeons: 30-day operative mortality and morbidity risk models. Ann Thorac Surg 2003;75:1856 – 64; discussion 1864 –5.
11. Welke KF, Shen I, Ungerleider RM. Current assessment of
mortality rates in congenital cardiac surgery. Ann Thorac
Surg 2006;82:164 –71.
12. O’Brien SM, Jacobs JP, Clarke DR, et al. Accuracy of the
Aristotle Basic Complexity Score for classifying the mortality
and morbidity potential of congenital heart surgery operations. Ann Thorac Surg 2007;84:2027–37.
13. Clarke DR, Breen LS, Jacobs ML, et al. Data verification in
congenital cardiac surgery. Cardiol Young 2008;18(suppl.
2):177– 87.
14. Elfstrom J, Stubberod A, Troeng T. Patients not included in
medical audit have a worse outcome than those included. Int
J Qual Health Care 1996;8:153–7.
15. Gibbs JL, Monro JL, Cunningham D, Rickards A. Survival
after surgery or therapeutic catheterisation for congenital
heart disease in children in the United Kingdom: analysis of
the central cardiac audit database for 2000 –1. BMJ 2004;328:
611.
16. Maruszewski B, Lacour-Gayet F, Monro JL, Keogh BE, Tobota Z, Kansy A. An attempt at data verification in the
European Association for Cardio-Thoracic Surgery Congenital Database. Eur J Cardiothorac Surg 2005;28:400 – 4.
17. Capps CS, Dranove D, Greenstein S, Satterthwaite M. The
silent majority fallacy of the Elzinga- Hogarty criteria: a
critique and new approach to analyzing hospital mergers.
Working paper. Washington, DC: National Bureau of Economic Research; 2001.
18. Spiegelhalter DJ. League tables. In: Armitage P, Colton T,
eds. Encyclopaedia of biostatistics. Chichester, UK: John
Wiley and Sons; 2005:2478 –751.
19. New York State Department of Health. Coronary artery
bypass surgery in New York State, 1997–9. Available at:
http://www.health.state.ny.us/nysdoh/heart/heartdisease.
htm. Accessed Oct 11, 2007.
20. Spiegelhalter DJ. Funnel plots for comparing institutional
performance. Stat Med 2005;24:1185–202.
21. Jacobs ML, Jacobs JP, Franklin RCG, et al. Databases for
assessing the outcomes of the treatment of patients with
congenital cardiac disease–the surgical perspective. Cardiol
Young 2008;18(suppl 2):101–15.
22. Jacobs JP, Jacobs ML, Mavroudis C, Lacour-Gayet FG, Tchervenkov CI. Executive summary: the Society of Thoracic
Surgeons congenital heart surgery database-seventh harvest- (2003–2006). Durham, NC: The Society of Thoracic
Surgeons and Duke Clinical Research Institute; 2007.
23. Jacobs JP, Jacobs ML, Mavroudis C, Lacour-Gayet FG, Tchervenkov CI. Executive summary: the Society of Thoracic
Surgeons congenital heart surgery database- eighth harvest(2004 –2007). Durham, NC: The Society of Thoracic Surgeons
and Duke Clinical Research Institute; 2008.
24. U.S. District Court for the District of Columbia. Consumers’
Checkbook, Center for the Study of Services v. U.S. Department of Health and Human Services, Civil action No. 06 –2201
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
686
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
ETHICS IN CARDIOTHORACIC SURGERY
PUBLIC REPORTING OF OUTCOMES
JACOBS ET AL
(EGS). August 22, 2007. Available at: http://www.checkbook.
org/Press/doc/Court%20Opinion.pdf. Accessed Apr 22, 2008.
Shahian DM, Silverstein T, Lovett AF, Wolf RE, Normand
SL. Comparison of clinical and administrative data sources
for hospital coronary artery bypass graft surgery report
cards. Circulation 2007;115:1518 –27.
Cronk CE, Malloy ME, Pelech AN, et al. Completeness of
state administrative databases for surveillance of congenital
heart disease. Birth Defects Res A Clin Mol Teratol 2003;67:
597– 603.
Frohnert BK, Lussky RC, Alms MA, Mendelsohn NJ, Symonik DM, Falken MC. Validity of hospital discharge data
for identifying infants with cardiac defects. J Perinatol 2005;
25:737– 42.
Strickland MJ, Riehle-Colarusso TJ, Jacobs JP, et al. The
importance of nomenclature for congenital heart disease:
implications for research and evaluation. Cardiol Young
2008;18(suppl. 2):92–100.
Sherry A. Children’s Hospital cardiology chief told to resign.
Denver Post March 1, 2001.
The Denver Post Editorial Board. At the heart of the problem. Denver Post March 2, 2001.
Committee on Quality of Health Care in America. To err is
human: building a safer health system 2000. Available at:
http://www.nap.edu/catalog.php?record_id⫽9728. Accessed
Apr 14, 2008.
National Quality Forum. Available at: http://www.
qualityforum.org/. Accessed Oct 11, 2007; Oct 12, 2008.
Centers for Medicare and Medicaid Services. http://www.
cms.hhs.gov/pqri/. Accessed Dec 7, 2008.
Improving the interoperability of health information technology. 110th Congress, 1st Session ed. United States. Washington, DC; 2007.
Personal communication. E-mail communication from:
Hedstrom, John, Asst. Director of Government Relations,
The Society of Thoracic Surgeons. To: Jeffrey Jacobs, MD.
Subject: RE: Latest Version of Senate HIT/Public Reporting
Legislation. Sent: Wednesday, October 31, 2007, 4:00 PM.
Ann Thorac Surg
2009;87:679 – 86
36. Porter ME, Teisberg EO. How physicians can change the
future of health care. JAMA 2007;297:1103–11.
37. Porter ME, Teisberg EO. Redefining health care: creating
value-based competition on results. Boston, MA: Harvard
Business School Press; 2006.
38. Jacobs JP. Caring for patients with congenital cardiac
disease - introduction to part 6 of the supplement. Cardiol
Young 2005;15(suppl 1):159 – 60.
39. Cohen M, Jacobs JP, Quintessenza JA, et al. Mentorship,
learning curves and balance. Cardiol Young 2007;17(suppl
2):164 –74.
40. Bonchek LI, Harley DP, Wilbur RH, et al. The STS future
planning conference for adult cardiac surgery. Ann Thorac
Surg 2003;76:2156 – 66.
41. Mayer JE. Threats to the medical profession by the American
Medicare system. Presented as a Visiting Professor at the
Ninth Annual Anthony RC Dobell Day in Congenital Cardiac Surgery, Montreal Children’s Hospital and McGill
University, Montreal, Canada; Dec 4, 2006.
42. The Reserves of life. St. Mary’s Hosp Gaz 1907;13:95– 8.
43. American College of Physicians. Avaialable at: http://
www.acponline.org/college/pressroom/osler_feat.htm. Accessed: Oct 11, 2007.
44. American College of Physicians. Editor of ‘The Quotable
Osler’ selects ten favorites 2003. Available at: http://
www.acponline.org/pressroom/osler_feat.htm. Accessed
Apr 14, 2008.
45. Shahian DM, Edwards FH, Ferraris VA, et al. Quality measurement in adult cardiac surgery: part 1–Conceptual framework and measure selection. Ann Thorac Surg 2007;83(4
suppl):S3–12.
46. O’Brien SM, Shahian DM, DeLong ER, et al. Quality measurement in adult cardiac surgery: part 2–Statistical considerations in composite measure scoring and provider rating.
Ann Thorac Surg 2007;83(4 suppl):S13–26.
47. Sade RM. Deceiving insurance companies: new expression
of an ancient tradition. Ann Thorac Surg 2001;72:1449 –53.
Downloaded from ats.ctsnetjournals.org by on July 11, 2011
The Ethics of Transparency: Publication of Cardiothoracic Surgical Outcomes in
the Lay Press
Jeffrey P. Jacobs, Robert J. Cerfolio and Robert M. Sade
Ann Thorac Surg 2009;87:679-686
DOI: 10.1016/j.athoracsur.2008.12.043
Updated Information
& Services
including high-resolution figures, can be found at:
http://ats.ctsnetjournals.org/cgi/content/full/87/3/679
References
This article cites 27 articles, 15 of which you can access for free at:
http://ats.ctsnetjournals.org/cgi/content/full/87/3/679#BIBL
Citations
This article has been cited by 4 HighWire-hosted articles:
http://ats.ctsnetjournals.org/cgi/content/full/87/3/679#otherarticles
Subspecialty Collections
This article, along with others on similar topics, appears in the
following collection(s):
Professional affairs
http://ats.ctsnetjournals.org/cgi/collection/professional_affairs
Permissions & Licensing
Requests about reproducing this article in parts (figures, tables) or
in its entirety should be submitted to:
http://www.us.elsevierhealth.com/Licensing/permissions.jsp or
email: [email protected].
Reprints
For information about ordering reprints, please email:
[email protected]
Downloaded from ats.ctsnetjournals.org by on July 11, 2011