SIMULATING THE NEED FOR RENAL REPLACEMENT THERAPY CAPACITY
Anders L Nielsen1), Alok Kumar1), Philip Nielsen2), Susanne Petersen1)
1) University of the West Indies, Cave Hill, Barbados 2) Act-Consult, London, UK
email: [email protected]
1. KEYWORDS
Discrete Event Simulation, Kidney failure, Model building,
Patient-centered simulation, Renal Replacement Therapy
capacity, Health Care optimization.
2. ABSTRACT
Barbados has experienced a rising burden of patients with
end-stage renal failure (ESRF) and the burdens of a renal
replacement therapy (RRT) on society are significant. We
developed a patient-centered discrete event simulation
(DES) model to simulate the needed RRT capacity fitted to
local conditions. Two scenarios were analyzed.
1) What happens over the next 5 years if the present
capacity is maintained assuming unchanged prevalence and
incidence of ESRF? The simulation indicated that
equilibrium with 300 patients receiving RRT will be
reached during the next 5 years. This equilibrium is best
explained by increased mortality due to rationing the
dialysis time.
2) What is the effect of improved predialysis treatment
without resource limitations? The simulation indicates that
one would expect a 19% increase in patients who survive
the first year. In this scenario it means 387 patients’ years
gained.
3. INTRODUCTION
Health care systems worldwide have seen a rising burden of
patients with Chronic Kidney Diseases (CKD) that for some
unfortunately will progress to end-stage renal failure
(ESRF) 1. When the patient reaches this stage the patient
needs renal replacement therapy (RRT) to stay alive. Only
three treatment modalities for RRT are available namely
renal transplantation (TX) and two forms of dialysis, either
peritonealdialysis (PD) or hemodialysis (HD).
The prevalence of patients with CKD in a particular region
is determined by one set of factors. An additional set of
factors influence how many of the patients with CKD will
ultimately progress to ESRF. The conditions that lead to the
majority of CDK cases are obesity, diabetes and
hypertension. But also race and comorbidities like HIV,
hepatitis and lupus are other cofactors that define the
number of patients that will need RRT and hence the health
care resources needed.
Regardless of which form of RRT the patient receives it is a
chronic and costly treatment. The burden of RRT on society
is significant and add to the already existing ‘health care
crisis’2. Consequently health care systems have to increase
efficiency3. The challenge then becomes how to improve the
outcomes for these patients within the given constraints?
One has to evaluate many possible scenarios to find the
most optimal ones. Obviously evaluations of different
interventions call for tools that can be used assisting with
the planning of future healthcare services to become more
efficient.
The aim of this paper is to share our considerations and
experiences when building a patient-centered discrete event
simulation (DES) model estimating the need for RRT
capacity. The ultimate goal is to develop a tool that can
contribute to the process of optimizing economic efficiency
in healthcare systems offering RRT to its population.
4. METHODS
We used the ARENA® simulation software (V 13.5 from
Rockwell Automation Technologies Inc., USA) to build our
simulation model and run the simulations. Initially we used
the free but limited student version due to economic
constraints. However that forced us to think in terms of
prototyping and building models that can be used as submodels. Previously we have used a prototyping approach
with success 4 and it is our experience that it is straight
forward to build models that consist of many sub-models
and eventually integrate them into one comprehensive
bigger model.
As literature reference database we use Refmann® (Version
12 from Thomson Reuters) and we have presently collected
almost 800 references with clinical information relevant for
building a RRT DES model.
To keep a dynamic graphical overview of the associative
relationships between the different clinical components we
use PersonalBrain Pro® (Version 6 from TheBrain
Technologies LP).
5. THE MODEL
5.1.
The
Macro Model
On the Macro level our simulation model assumes the same
universal RRT model as originally presented by Davies &
Roderick5. It basically deals with the flow within what we
could call the ‘Chronic RRT-core’. Patients with ESRF in
need of RRT enter the model based on known crude
incidence rates. The model also simulates interaction
between the three available treatment-modalities (PD, HD,
TX) based on known transfer frequencies between the
modalities. There is by nature only one-way out of the
model (i.e. death) determined by the crude mortality rates.
The strengths of Davies & Roderick’ model5 is its simplicity
and universality on the macro level. It only needs changing
a few crude rates to refit the model to any country/region
offering RRT services.
5.2.
The
module ‘Chronic HD’
For socio-economic reasons the main RRT modality used in
Barbados is HD. Accordingly the model was fitted to
simulate the HD modality more accurately (for an overview
of our final model see Figure 1).
The macro model by Davies & Roderick5 does not simulate
each single HD session and the model among other things
does not take into consideration what happens when the
number of patients in need of HD exceeds the present
capacity. Hence the HD modality in the macro model was
expanded with the sub-module ‘Chronic HD’ where each
single HD session is simulated.
For more than four decades, the standard worldwide
schedule for hemodialysis has continued to be three sessions
a week, largely owing to logistic and cost concerns6 The
module simulates conventional thrice-weekly hemodialysis
sessions of 2.5 to 4.0 hours7 duration, with median set to 3
Figure 1 Model overview
hours. The simulated facility has 24 HD machines
(likelihood of break down included) and they operate on a
double shift schedule six days a week. We have at present
not expanded the model with the option of the patients
receiving more frequent HD sessions than thrice weekly as
the evidence of benefits with more than thrice-weekly HD
sessions is conflicting6-8.
If the numbers of persons in need of RRT exceed the
available slots some patients already on HD are reduced to
receive only 4 hours HD twice weekly. As many patients as
needed are put on reduced HD to free slots for the new
patients. If capacity is maximized by using the 2 times 4
hour HD regimen using all allocated resources, no new
patients are accepted. The 2 times weekly regimen carries a
price in terms of increased mortality and morbidity
compared to the longer 3 times weekly option hence the
model incorporates different mortality and morbidity for the
two treatments9. If slots become available the model accepts
patients transferred from other RRT modalities as first
priority, new patients as second priority and finally as third
priority patients on 2 times weekly HD are increased to 3
times weekly.
The sub-module ‘Chronic HD’ (see Figure 1) is further
divided in three sub-modules. The first step is that the
model checks the existing capacity (see Figure 2). If there is
a slot available for HD 3 times weekly the patient is
transferred to the sub-module ‘HD 3 times weekly’ (See
Figure 3). If there are no slots available the patient is
transferred to the sub-module ‘HD 2 times weekly’ (see
Figure 4). The reason for this construction is of cause the
difference in mortalities and morbidities and we want to
keep track of which weekdays the patients receive HD.
patients with immediate need for RRT but waiting for
treatment. This is mathematically a straightforward module
(see Figure 5). If the patient does not receive dialysis (i.e.
accepted by the Acute HD module) within a short time
frame the outcome is death (i.e. the item exists in the model
only while alive and in queue).
Figure 5 In need of HD but waiting.
5.4.
Figure 2 Submodel checks capacity
The
module ‘Acute HD’
It is well known that mortality and morbidity are higher
during the first three months on HD10. An issue not
addressed by the ‘chronic HD’-core module since it does not
specifically simulate the first 3-months on HD. The added
‘Acute HD’ module simulates the increased mortality and
morbidity that is found during the first 3 months. The submodule is similar to the ‘chronic HD’ sub-module but with
increased mortality rate. A rate that in simple terms is
dependent on how stigmatized the patients are and the
existence of comorbidities.
5.5.
Figure 3 HD three times a week
Figure 4 HD two times a week
5.3.
The
module “In need waiting for HD”
As a consequence of the occurrence of cases where no HD
slot is available a sub-module had to be added simulating
Entr
y
Meeting a number of KDOQI guideline goals at dialysis
initiation is independently associated with survival during
the first year of dialysis treatment. These goals are primarily
related to three issues: That the patient has a permanent
vascular access before starting on HD, has the CKD related
anemia well treated and has a good nutritional status11, 12.
The model therefore has 4 entry points. One called ‘timely
referrals’ for patients who have been prepared and are ready
for HD meeting all three goals. They enter directly into the
sub-module ‘Chronic HD’ as they have no initial increased
mortality. The second entry point is the ‘late referrals’ for
the patients who meet two goals. The third is the ‘known not
ready’ for patients who only meet one goal and the fourth
entry point the ‘acute’ for the ones who has no preparation
at all (not shown in Figure 1). Unfortunately most of our
patients fit the ‘acute’ category. The entry points are
modeled with increased mortality risk initially and the
patients are forwarded to the ‘in need waiting for HD’
module except for ‘timely referrals’ as already mentioned
they go directly to the ‘chronic – HD’ module.
The
‘Start up’ module
Initially the model did not behave as expected. We had very
few on HD and hardly any mortality during the first ½ year
and then it suddenly rose significantly. We had made the
classic mistake of omitting a run-in period13. We then
finally added a module that feeds the current stock of
patients into the model during the first week of simulation.
5.7.
The
PD Module
We have added a PD module. At present we have very few
patients on PD simply because the social environment does
not facilitate the use of PD as a viable RRT option. And the
few put on PD generally after a short time is transferred
from PD to HD on medical grounds, but the underlying
cause is primarily cultural-social-economical.
not receive sufficient treatment (the ones on twice weekly
HD). On the other hand from a health care system
perspective it can be claimed that with the current allocated
resources at least 300 patients will receive some treatment.
However such ethical dilemmas can never be solved using
simulation – we can only contribute to the quest to become
more efficient.
6. RESULTS
6.1.
Wha
t is the present capacity?
Simulation of the model was performed with 1000
replications of a 5 year period. With the initial number of
patients on HD (approximately 200 in 2010) and the initial
rate of admissions, the simulation predicted a larger capacity
than perceived by the staff. The simulation indicated that the
capacity was only utilized to 80.2 % of its scheduled
capacity. Organizational and cultural barriers are believed to
be major contributors to the non-optimal use of the
resources.
Model group
5.6.
Average
Avg min
Avg Max
Worst case
scenario #
424
New
Patients
300
196
424
with ESRF
Died waiting for
3.5
0
20
0
HD
Died
first
3
84
11
138
11
months on HD
Died on Chronic
223
136
304
136
HD
Total 5 year gain
-11
49
-38
277
Net yearly gain
--2
9.8
-8
55
# is inversed worst case scenario i.e. number of incoming maximized,
number of death minimized.
Table 1: Results from five years simulations
The simulation reached a steady state with a population on
RRT of approximately 300. However it also showed the
possibilities of peaks where the current resources would not
meet demand, as seen by the fact the average maximum for
patients dying while waiting for treatment is 20 [see Table
1].
On the one hand from a patient perspective this steady state
situation is unacceptable. It is reached because some
patients die waiting for HD while others die because they do
6.2.
Wha
t if predialysis treatment is improved?
An additional question is what happens if we have increased
resources (i.e. no patients die waiting and all patients are on
trice weekly HD) and at the same time improve the
predialysis treatment so that a larger percentage of patients
will enter the model with the label ‘timely referrals’. For
details of such an intervention see Table 2.
No of
criteria
met
Mortality
hazard
ratios
(95%
confidence
intervals)
Percent of new patient
( N yearly = 70)
Pre-intervention
Post-intervention
Timely referrals
3
0.34
(0.30 -0.39)
5%
60 %
Known not
ready
2
0.53
(0.51 -0.56)
10 %
25 %
Late referrals
1
0.81
(0.80 -0.83)
25 %
10 %
Acute referrals
0
1
60 %
5%
Table 2 Improved predialysis treatment
With the changes in the referrals (i.e. post interventions
data) the simulation is repeated over 5 years with 500
iterations. The utility of the HD machines is initially 64%
(see Figure 6). In year one in both the pre- and postinterventions scenarios the system needs to work at its
present maximum allocative efficiency (Usage 66.5% = two
full shifts). It is very unlikely that the initial capacity is
sufficient after year one. (The minimum average usage is
66.8%, average is 71.0% and maximum usage is 75.2%).
Data Collection, Model Building, Verification and
Validation, Analysis, Simulation Graphics, Managing the
Simulation Process, and Human Factors, Knowledge and
Abilities. Only Data Collection, Model Building, Analysis,
and Human Factors, Knowledge and Abilities are of special
interest for our project.
7.2. Data collection
Figure 6 HD Machine utility before and after improvement to
pre-dialysis treatment
Five year after the intervention on average 38.7 more
patients will have survived the first year on RRT. That is an
increase of 19.4% in patients who survive the first year With
an average life expectancy of 10 years on HD the
intervention therefore is predicted to gain 387 years of life.
It means that one year after the intervention has been
implemented; on average 9.67 additional patients annually
will survive the first year of RRT.
7. DISCUSSION
7.1. Why choose DES
Why choose DES? Literature reviews reveal that
simulations are the prominent method used for planning and
system/resource utilization projects in health care 14, 15.
Four characteristics have to be reflected in the model. First
it deals with individual patients. Secondly the model has to
be a Non-Markovian since the future (e.g. survival on HD)
for the individuals is conditionally dependent of the past
(e.g. the patient is diabetic), given the present (i.e. the
patient has reached ESRF). Thirdly the individuals belong
to different discrete-states (i.e. different disease stages
modified by different comorbidities) and finally the model
examines interactions with the environment (e.g. availability
of resources such as staff and equipment and/or available
treatment modalities). Applying the taxonomy proposed by
Brennan et al. 16 suggests that DES is an optimal method.
Since there is a relatively small volume of literature on what
constitutes good practice and it is not prescriptive about
every aspect of decision modeling Sculpher et al.17 suggest
the analysts provide explicit and comprehensive justification
of their methods instead, and allow the user of the model to
make an informed judgment about the relevance, coherence
and usefulness of the analysis. Banks and Chwif’s 18
proposed to describe the process using seven categories:
7.2.1. Data format - Applying clinical risks to the
model
Data is often in the wrong format, collected as discrete data
rather than continuous data 18. Sod’s law of data collection
says that the data available is never quite exactly the data
you want because it was originally collected for a purpose
different from your simulation study 18. In the medical
literature clinical evidence is often found in reported
randomized trials and systematic reviews and they generally
report on dichotomous outcomes (such as death versus
survival)19 . Binary variables are easily modeled in a DES –
it is either state A or state B – so the converting of clinical
evidence into DES-models seems straight forward.
The task for the modeler is the same as the physician faces.
For the physician the question is how is data from literature
interpreted and converted into meaningful advice for the
individual patient 20. For the modeler the task is to translate
the data from literature into probabilities determining the
fate of the entity.
When building simulation models in the medical domain the
modeler needs to extract knowledge from several studies
that have reported on the same outcomes and that is where
things starts to be a little complicated for the modeler. The
outcomes can be reported using several different
terminologies including hazard ratios, absolute risks,
relative risks, risk ratios and odds ratios. Our literature
database at present consists of 251 publications describing a
clinical investigation relevant for the model. The number of
different probability terms used in the publications is shown
in Table 3. Note that 185 of the publications only use a
single term but 66 use a combination of terms.
Absolute Risk (AR), Relative Risk or Risk Ratio (RR)
values can be directly applied to a DES model but in our
case they only represent a quarter of the 251 publications.
For the remaining three quarters of publications using
Hazard ratio (HR), Kaplan-Meier (KM) and Odds Ratio
(OR) the challenge for the modeler is to convert the data
into probabilities that can be applied to the model.
Absolute risk (AR)
Relative risk (RR)
Risk ratio (RR)
Odds Ratio (OR)
AR
1
4
0
0
RR
4
37
2
7
RR
0
2
1
1
OR
0
7
1
36
HR
1
3
0
6
KM
0
2
0
0
Sum
6
55
4
50
Hazard ratio (HR)
Kaplan-Meier
(KM)
Sum
1
0
3
2
0
0
6
0
93
7
7
17
110
26
6
55
4
50
11
0
26
251
Table 3 Method used to describe risk in the literature database.
7.2.2. Hazard Ratio
Survival is often reported at intervals as Hazard ratio (HR),
i.e. as discrete data even though they in reality represent a
continuous dataset. HR is a measure of how often a
particular event happens in one group compared to how
often it happens in another group, over time (t). HR is the
most complicated to convert to the model since it is a
measure of Relative Risk - also called Risk Ratio- (RR) over
t. Hence it is used not only to describe the total number of
events, but their timing as well. The HR equals a weighted
RR over the entire duration of a study and is often derived
from a time-to-event curve or Kaplan-Meier plot 21.
Especially two problems related to HR need to be
mentioned. First the timing of events may not be evenly
distributed over time throughout the study period and many
studies report only a single HR averaged over the duration
of the study’s follow up 22. The challenge for the modeler is
to subdivide t into smaller intervals that make sense. If a
Kaplan-Meier plot is presented in a publication a feasible
way is to read the values from the Kaplan-Meier plot at the
required intervals. In our literature database only 7 of 110
(see Table 1) describing HR’s presented a Kaplan-Meier
plot, therefore the only other option is to have a medical
expert estimating the characteristics of the HR-function over
time.
Secondly the period-specific HR has a built-in selection
bias. To describe this bias, consider that the (discrete-time)
hazard during period t is defined as the risk of the outcome
during period t among those who reached period t free of the
outcome 22. Thus the hazard ratio must be interpreted
judiciously especially in settings where the duration of
events or the disease are the primary efficacy variables 23.
7.2.3. Risk and odds ratio
Clinicians find it difficult to understand odds and odds
ratios as measures of association, although they may be
comfortable with the parallel concepts of risk and risk
ratios.
Although both the risk ratios and odds ratios are perfectly
valid ways of describing a treatment effect, it is important to
note that they are not the same measures, cannot be used
interchangeably and should not be confused. For treatment
that increases the chance of an event, the odds ratio will be
larger than the risk ratio. For interventions that reduce the
chance of an event, the odds ratio will be smaller than the
risk ratio. Thus if an odds ratio is misinterpreted as a risk
ratio it will lead to an overestimation of the intervention.
Unfortunately, this error and interpretation is quite common
in published reports on individual studies and systematic
reviews 24. The bottom line is: When calculations have been
based on odds ratios the modeler has to transform the
findings to the concept of risk.
7.3. Model building
We create first a conceptual model prior to the
implementation of the computerized model using
PersonalBrain Pro®. A conceptual model is an abstraction
of the real system that is being studied. The conceptual
model consists of 18:
1) Assumptions on system components.
2) Structural assumptions that define the interactions
between system components. These are expressed by means
of natural language and diagrams and
3) Input parameters and data assumptions.
It is recommended that model structure should be as simple
as possible and only with complexity level as needed 14, 18.
The single dominate error in logic modeling is to
incorporate excessive details 25. In addition the model
should be consistent with the stated decision problem and it
should be based on theory of the diseases and not just
defined by data availability or health service inputs alone 16.
Therefore the model described in this paper is simple but
complex enough to reflect the dynamics and characteristics
of patients with ESRF as well as the interactions between
the patients and the health care system that provide the
services the patients need to stay alive.
7.4. Analysis
Our simulation model has most features common with a
‘push system’ and few features common with a ‘pull
system’
A push system has external arrivals at some arrival rate (i.e.
entities are pushed into the system such as patients
categorized as ‘late arrivals’). A pull system demands
entities to feed it (i.e. entities are pulled into the system such
as patients categorized as ‘timely referrals’). Obviously the
kidney model is mainly a push system. Patients need
dialyses when they do - but ‘timely referrals’ can be argued
to attain some features of the pull system. 18
7.5. Human factors (Knowledge and abilities)
The most critical component for a simulation project is not
software. Neither is it hardware. It is ‘human ware’. Beware
of the SINSFIT principle: Simulation is no substitute for
intelligent thinking 18.
As Banks stresses the key to successful models is the human
ware. A team with members whose qualifications and
competences are complementary to each other is an asset.
The combination of expertise on kidney diseases, research
interpretations, public health, human behavior, system and
interactive analysis and software programming definitely
facilitated the development of our model.
7.6. Ethical issues.
During the development of the Chronic HD module it
became clear that during certain periods the need for HD
exceed the capacity provided using trice weekly HD
sessions. We had to investigate how the situations were de
facto handled when demand exceeded the capacity since no
formal policy existed and we found no “hidden capacity". It
became evident that an informal priority setting/rationing
took place. In order to make capacity available for new
patients the amount of HD given to patients already in the
program was reduced. This informal priority setting had to
be incorporated into the model (see Figure 2 Submodel
checks capacity). One could argue that it hereby became an
explicit priority setting (i.e. a described rule) at least in the
model.
This example serves as a reminder of the fact that the
process of building DES models not only might aid in future
planning but also is a tool to clarify which processes really
take place in the organization. We will argue that this case
once again shows that DES can be a valuable tool when
optimizing complex adaptive healthcare systems4.
8. CONCLUSION
We have presented a patient-centered Discrete Event
Simulation model that simulates the need for Renal
Replacement Therapy capacity. The model simulates each
individual dialysis session and it assigns different
mortalities to the individual patient groups. The model has
been used to illustrate two scenarios.
The first scenario simulates the number of patients that the
system can maximally cope with over a five year period
with unchanged resources. It shows that with a rationing
procedure put in place (i.e. decreased dialysis) the mortality
increases and hence a new steady state level is reached.
The second scenario simulates the consequences of an
intervention aimed at improving pre-dialysis treatment
without resource limitations. It shows a significant number
of years of survival gained if this intervention was
introduced.
We have highlighted the crossroads that we have
encountered and the considerations and decisions that the
team had to handle. In this process it was an asset having
team members with qualifications and competences in
different domains.
Despite the barriers and the complexity we still argue that it
is possible to build relatively simple models that can be used
in the processes aiming at optimizing health care systems.
Reference List
1. Meguid El NA, Bello AK. Chronic kidney disease: the
global challenge. Lancet 2005;365(9456):331-340.
2. Goldsmith JC. The new health-cost crisis. Harv Bus
Rev 2001;79(10):20-21.
3. WHO. World Report on Knowledge for Better Health
- Strengthening Health Systems. Geneva,Switzerland:
World Health Organization, 2004
4. Lassen Nielsen A, Hilwig H, Kissoon N,
Teelucksingh S. Discrete event simulation as a tool in
optimization of a professional complex adaptive
system. Stud Health Technol Inform 2008;136:247252.
5. Davies R, Roderick P. Predicting the future demand
for renal replacement therapy in England using
simulation modelling. Nephrol Dial Transplant
1997;12(12):2512-2516.
6.
Himmelfarb J, Ikizler TA. Hemodialysis. New
England Journal of Medicine 2010;363(19):18331845.
7. Chertow GM, Levin NW, Beck GJ et al. In-Center
Hemodialysis Six Times per Week versus Three
Times per Week. New England Journal of Medicine
2010;363(24):2287-2300.
8. Wheeler DC, Caplin B. New Observational Data
Demonstrate that Mortality Is Lower in Patients
Receiving More Frequent Dialysis. Journal of the
American Society of Nephrology 2012.
9. Saran R, Bragg-Gresham JL, Levin NW et al. Longer
treatment time and slower ultrafiltration in
hemodialysis: associations with reduced mortality in
the DOPPS. Kidney Int 2006;69(7):1222-1228.
10. Winkelmayer WC, Owen WF, Levin R, Avorn J. A
Propensity Analysis of Late Versus Early
Nephrologist Referral and Mortality on Dialysis.
Journal of the American Society of Nephrology
2003;14(2):486-492.
11. Slinin Y, Guo H, Gilbertson DT et al. Meeting
KDOQI guideline goals at hemodialysis initiation
and survival during the first year. Clin J Am Soc
Nephrol 2010;5(9):1574-1581.
12. Roubicek C, Brunet P, Huiart L et al. Timing of
nephrology referral: influence on mortality and
morbidity. Am J Kidney Dis 2000;36(1):35-41.
13. Kelton WD, Sadowski RP, Sturrock DT. Simulation
with Arena. 4 ed. New York: McGraw-Hill; 2007.
14. Jun JB, Jacobson SH, Swisher JR. Application of
Discrete-Event Simulation in Health Care Clinics: A
Survey. The Journal of the Operational Research
Society 1999;50(2):109.
15. Brailsford SC, Harper PR, Patel B, Pitt M. An
analysis of the academic literature on simulation and
modelling in health care. J of Sim 2009;2009(3):130140.
16. Brennan A, Chick SE, Davies R. A taxonomy of
model structures for economic evaluation of health
technologies. Health Econ 2006;15(12):1295-1310.
17. Sculpher M, Fenwick E, Claxton K. Assessing
quality in decision analytic cost-effectiveness
models. A suggested framework and example of
application. Pharmacoeconomics 2000;17(5):461477.
18. Banks J, Chwif L. Warning about simulation. Journal
of simulation 2011;2011(5):279-291.
19. Scott I. Interpreting risks and ratios in therapy trials.
Aust Prescr 2008;31:12-16.
20. Roderick PJ. Assessing the impact of chronic kidney
disease on individuals and populations: use of
relative and absolute measures. Nephrol Dial
Transplant 2012.
21. Boslaugh S, Watters PA. Statistics in a nutshell. 1 ed.
Cambridge: O'Reilly; 2008.
22. Hernan MA. The hazards of
Epidemiology 2010;21(1):13-15.
hazard
ratios.
23. Spruance SL, Reid JE, Grace M, Samore M. Hazard
ratio in clinical trials. Antimicrob Agents Chemother
2004;48(8):2787-2792.
24. Center for Reviews and Dissemination. Systematic
Reviews. CRD's guidance for undertaking reviews in
healthcare. CRD, University of York; 2009.
25. Schmeiser BW. Some Myths and Common Errors in
Simulation Experiments. Proceedings of the 2001
Winter Simulation Conference 2012;39-46.