Supplemental Materials
Participants
Subjects seen in the ER (n=270) did not differ from a larger cohort of 1436 consecutive
trauma admissions to the same ER in age (31.2±10.9 vs. 32.7±11.0), gender distribution (58%
vs. 53% males, χ2=1.48, n.s.) and trauma types indicating that those recruited for this study
represented the broader ER population.
One hundred eighty-two out of 270 subjects seen in the ER have agreed to receive a
telephone call in view of further participation in the study, and 163 of the latter (89.5%) attended
the first follow-up visit. Of those attending the initial session n=155 (95.1%) completed the
study. Three study completers had less than two assessments and were excluded from
longitudinal trajectories computation, leaving a current sample of n = 152.
Biological samples and their analyses
Samples for hormonal analyses were collected following subjects' consent in the ER and
upon their arrival to the hospital on subsequent sessions. Blood samples were spun immediately
in a cold centrifuge, and frozen for subsequent analysis. For GR analyses, 25 ml plasma were
obtained and mononuclear cells were isolated using Ficoll Hypaque within 1 h following blood
draw. Cells were centrifuged at 300 g at 4°C, washed four times in ice-cold Hank's buffer, and
pelleted. An aliquot of the suspension was counted by a haemocytometer and the final pellet
stored at −70°C.
Saliva samples were collected, at the same time, in two salivettes, and preserved at
−40°C. Urine samples were collected (a) during 4 h in the ER (initial void and subsequent
collection) and (b) at the beginning of each follow-up assessment session (single void). The time
since last void was recorded for each urine collection. Hourly urine excretion was determined by
multiplying the concentration by the volume and then dividing by the time since last void.
Laboratory tests were performed at James J. Peters Bronx Veterans Affairs (Dr. Yehuda's
laboratory) according to previously published protocols (1, 2). Biological samples were frozen
and sent via courier, on dry ice. Samples were analyzed upon arrival at the laboratory. The
laboratory personnel were blind to subjects' identities, time since trauma and diagnostic status.
Latent Growth Mixture Modeling Procedure
Latent Growth Mixture Modeling (LGMM; 28) was employed to identify PTSD
symptom severity trajectories (IES-R total scores) from 10 days to 5 months using MPlus 7.2 (3).
The best fitting model was selected through a nested model approach where progressive numbers
of classes are fit until model fit indices no longer favor additional classes (see supplemental
information for statistical selection criteria). The best fitting model was determined using a
confluence of evidence across conventional indices, including reductions in the Bayesian
Information Criterion (BIC), sample-size adjusted Bayesian Information Criterion (SSBIC),
Aikaike Information Criterion (AIC) indices, and significance indicated by the Lo-MendellRubin likelihood Ratio test (LRT), the Vuong-Lo-Mendell-Rubin Likelihood Ratio Test (VLRT),
and the Bootstrap Likelihood Ratio Test (BLRT), along with parsimony and interpretability
equally weighed consistent with recommendations from the literature (4). Entropy was utilized to
determine the clarity of class specification with scores closer to one indicating better fit of the
data into the prescribed class structure (5). Individuals were assigned to classes based on the
highest posterior probability of class membership in identified latent trajectories. Class
membership was used for analysis as the outcome variable in subsequent machine learning
predictive and graph analyses. The predictive accuracy of trajectories was compared to PTSD
diagnostic status established through clinician assessment using the CAPS.
Data Preparation for Predictive and Graph Modeling
Prior to analysis, categorical variables were dummy coded and continuous variables were
normalized to range [0-1]. Variables with >30% missing data were not included in analyses.
Causal analyses included data that was complete for all pairwise or conditional correlations for a
given set of interactions. To test if the pattern of missing data in the remaining features was
predictive of PTSD trajectory classes, predictive models were constructed with these features
where missing values were encoded as 0 and non-missing values were encoded as 1. The
resulting predictive performance (AUC) was 0.49 indicating that the missingness in these
features met the missing at random assumption. Therefore, the following encoding was
implemented to handle the missing data: (1) all non-missing values in individual features were
linearly scaled between 0 and 1; (2) all missing values in individual features were set to -1.
Predictive Modeling
First, data was transformed and prepared for modeling (see supplemental Materials for
full description of procedures). Next, multiple models were built with data from progressive time
points integrated to determine the accuracy of predicting trajectories. Specifically, predictive
models were constructed including a) pre-trauma exposure variables only (i.e. demographics
including age, gender, prior trauma history etc.); b) pre-trauma + ER data; c) pre-trauma, + ER +
1 week data; d) pre-trauma + ER + 1 week + 1 month data. All models were also constructed on
1) clinical/demographic data alone; 2) neuroendocrine data alone; 3) clinical/demographic and
neuroendocrine data together
Accuracy metrics and Guards against Over-fitting
Predictive model results were subjected to 5x10 fold cross validation procedures to guard
against over-fitting and overestimation of prediction error. In this procedure, the best model is
identified in a randomly selected 4/5th of the data and tested in a hold-out 1/5th of the data. Fivefold cross validation was conducted ten times, each time with new random splits of the data to
prevent solutions that were driven by a sub-set of the sample resulting in a mean accuracy
represented as an average number of features selected and a mean area under the receiver
operator characteristic (ROC) curve (AUC) indicating the predictive accuracy of the variable set.
The ROC curve is a plot of the sensitivity versus 1-specificity of a classifier, and infers the
accuracy of that system, thereby creating a comparable metric across experiments (6). Following
convention (7, 8), AUC of .50-.60 indicates prediction at chance; .60-.70: poor prediction; .70–
.80: fair prediction; .80-.90: good prediction; .90–1.0: excellent prediction.
Table 1: Specificity at different sensitivity thresholds
Sensitivity Threshold
Feature Type
All
All
All
All
All
Endocrine
Endocrine
Endocrine
Endocrine
Other
Other
Other
Other
Time Point
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.93
0.93
0.84
0.84
0.73
0.69
0.56
0.47
0.29
(0.91,0.96)
(0.81,0.87) (0.81,0.87) (0.69,0.77) (0.64,0.73) (0.51,0.62) (0.41,0.53) (0.23,0.36)
Background (0.91,0.96)
0.97
0.97
0.94
0.94
0.88
0.86
0.80
0.75
0.62
(0.96,0.98)
(0.96,0.98)
(0.92,0.96) (0.92,0.96) (0.86,0.91) (0.83,0.89) (0.77,0.83) (0.71,0.79) (0.54,0.69)
Up to ER
1.00
1.00
0.99
0.99
0.97
0.95
0.90
0.84
0.68
(0.99,1.00)
(0.99,1.00)
(0.99,1.00) (0.99,1.00) (0.96,0.98) (0.94,0.97) (0.86,0.94) (0.78,0.89) (0.61,0.76)
Up to W1
1.00
1.00
1.00
1.00
0.99
0.98
0.94
0.91
0.83
(1.00,1.00)
(1.00,1.00)
(0.99,1.00) (0.99,1.00) (0.98,0.99) (0.97,0.99) (0.92,0.96) (0.87,0.94) (0.77,0.88)
Up to M1
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
(1.00,1.00)
(1.00,1.00)
(1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00)
Up to M4
0.93
0.93
0.85
0.85
0.72
0.67
0.58
0.52
0.39
(0.90,0.95)
(0.90,0.95)
(0.82,0.89) (0.82,0.89) (0.67,0.77) (0.62,0.72) (0.53,0.64) (0.46,0.58) (0.33,0.45)
Up to ER
0.91
0.91
0.83
0.83
0.75
0.69
0.56
0.50
0.36
(0.88,0.94)
(0.88,0.94)
(0.79,0.87) (0.79,0.87) (0.70,0.80) (0.63,0.75) (0.50,0.63) (0.43,0.56) (0.30,0.42)
Up to W1
0.95
0.95
0.88
0.88
0.77
0.71
0.62
0.56
0.44
(0.93,0.97)
(0.93,0.97)
(0.84,0.92) (0.84,0.92) (0.72,0.81) (0.66,0.76) (0.56,0.68) (0.50,0.62) (0.38,0.49)
Up to M1
0.95
0.95
0.91
0.91
0.83
0.80
0.71
0.65
0.55
(0.94,0.97)
(0.94,0.97)
(0.89,0.93) (0.89,0.93) (0.80,0.86) (0.76,0.84) (0.65,0.76) (0.59,0.71) (0.49,0.61)
Up to M4
0.96
0.96
0.93
0.93
0.85
0.83
0.70
0.64
0.49
(0.95,0.98)
(0.95,0.98)
(0.91,0.95) (0.91,0.95) (0.82,0.88) (0.80,0.86) (0.63,0.77) (0.57,0.71) (0.41,0.57)
Up to ER
0.99
0.99
0.98
0.98
0.94
0.92
0.86
0.82
0.75
(0.99,1.00)
(0.99,1.00)
(0.97,0.99)
(0.97,0.99)
(0.91,0.96)
(0.90,0.94)
(0.82,0.89)
(0.78,0.86)
(0.70,0.80)
Up to W1
1.00
1.00
0.98
0.98
0.96
0.94
0.89
0.85
0.74
(0.99,1.00)
(0.99,1.00)
(0.97,0.99) (0.97,0.99) (0.94,0.97) (0.91,0.96) (0.86,0.93) (0.81,0.89) (0.68,0.80)
Up to M1
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
(1.00,1.00)
(1.00,1.00)
(1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00) (1.00,1.00)
Up to M4
Values in individual cells represent the mean specificity and 95% confident interval computed over the 5*10 cross validation runs
0.9
0.21
(0.16,0.27)
0.57
(0.50,0.65)
0.60
(0.52,0.68)
0.74
(0.68,0.81)
1.00
(1.00,1.00)
0.33
(0.27,0.39)
0.29
(0.23,0.35)
0.38
(0.33,0.44)
0.49
(0.42,0.55)
0.41
(0.34,0.48)
0.67
(0.62,0.73)
0.68
(0.62,0.74)
1.00
(1.00,1.00)
1
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
0.00
(0.00,0.00)
Table 2: Positive Predictive Validity (PPV) calculated at different thresholds of sensitivity based on the precision-recall curve
Sensitivity Threshold
Feature Type
Time Point
All
Background
All
Up to ER
All
Up to W1
All
Up to M1
All
Up to M4
Endocrine
Up to ER
Endocrine
Up to W1
Endocrine
Up to M1
Endocrine
Up to M4
Other
Up to ER
Other
Up to W1
Other
Up to M1
Other
Up to M4
0.2
0.27
(0.21,0.33)
0.56
(0.47,0.65)
0.87
(0.81,0.93)
0.96
(0.92,1.00)
1.00
(1.00,1.00)
0.31
(0.24,0.37)
0.31
(0.24,0.38)
0.47
(0.37,0.56)
0.43
(0.35,0.51)
0.48
(0.40,0.56)
0.77
(0.69,0.84)
0.78
(0.70,0.86)
1.00
(1.00,1.00)
0.3
0.27
(0.21,0.33)
0.56
(0.47,0.65)
0.87
(0.81,0.93)
0.96
(0.92,1.00)
1.00
(1.00,1.00)
0.31
(0.24,0.37)
0.31
(0.24,0.38)
0.47
(0.37,0.56)
0.43
(0.35,0.51)
0.48
(0.40,0.56)
0.77
(0.69,0.84)
0.78
(0.70,0.86)
1.00
(1.00,1.00)
0.4
0.26
(0.22,0.30)
0.48
(0.42,0.55)
0.78
(0.71,0.84)
0.88
(0.83,0.93)
1.00
(1.00,1.00)
0.27
(0.22,0.32)
0.29
(0.25,0.33)
0.32
(0.27,0.36)
0.38
(0.32,0.44)
0.41
(0.36,0.47)
0.64
(0.58,0.71)
0.72
(0.65,0.79)
1.00
(1.00,1.00)
0.5
0.26
(0.22,0.30)
0.47
(0.40,0.53)
0.74
(0.68,0.81)
0.84
(0.79,0.90)
1.00
(1.00,1.00)
0.25
(0.22,0.28)
0.28
(0.24,0.32)
0.29
(0.25,0.33)
0.38
(0.32,0.43)
0.41
(0.35,0.46)
0.62
(0.55,0.68)
0.67
(0.61,0.74)
1.00
(1.00,1.00)
0.6
0.23
(0.20,0.26)
0.42
(0.36,0.47)
0.68
(0.61,0.75)
0.75
(0.69,0.82)
1.00
(1.00,1.00)
0.24
(0.21,0.28)
0.25
(0.21,0.29)
0.27
(0.23,0.31)
0.34
(0.29,0.39)
0.36
(0.31,0.41)
0.53
(0.47,0.60)
0.62
(0.55,0.69)
1.00
(1.00,1.00)
0.7
0.21
(0.19,0.24)
0.39
(0.34,0.44)
0.59
(0.52,0.67)
0.70
(0.62,0.77)
1.00
(1.00,1.00)
0.23
(0.21,0.26)
0.24
(0.20,0.27)
0.26
(0.22,0.29)
0.31
(0.27,0.35)
0.33
(0.28,0.37)
0.51
(0.44,0.57)
0.57
(0.50,0.64)
1.00
(1.00,1.00)
0.8
0.19
(0.17,0.21)
0.33
(0.29,0.38)
0.45
(0.37,0.52)
0.60
(0.52,0.67)
1.00
(0.99,1.00)
0.21
(0.19,0.23)
0.21
(0.18,0.23)
0.22
(0.20,0.25)
0.28
(0.25,0.32)
0.28
(0.24,0.33)
0.45
(0.39,0.51)
0.47
(0.40,0.54)
1.00
(0.99,1.00)
0.9
0.18
(0.16,0.20)
0.33
(0.28,0.37)
0.38
(0.32,0.45)
0.51
(0.44,0.58)
1.00
(0.99,1.00)
0.20
(0.18,0.22)
0.20
(0.18,0.22)
0.22
(0.20,0.24)
0.27
(0.24,0.29)
0.25
(0.21,0.29)
0.40
(0.35,0.45)
0.43
(0.36,0.49)
1.00
(0.99,1.00)
1
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
0.16
(0.16,0.17)
Figure 1:
Comparison of Four Best in Class Classification Algorithms to Linear SVMs with and without
Recursive Feature Elimination.
1.00
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
Linear SVMs: All Features
Linear SVMs: RFE
Polynomial SVMs: All Features
Polynomial SVMs: RFE
Gaussian BLR: All Features
All Features
Neuroendocrine
Features
Through M5
Through M1
Through W1
Through ER
Through M5
Through M1
Through W1
Through ER
Through M5
Through M1
Through W1
Through ER
Pre ER
Gaussian BLR: RFE
Laplace BLR: All Features
Laplace BLR: RFE
Random Forests: All Features
Random Forests: RFE
NonNeuroendocrine
Features
Note: Lines in the bar graph represent mean Area Under the Receiver Operator Characteristic
Curve (AUC) based on a) all available features; b) neuroendocrine features alone; c) all features
not including neuroendocrine features. Multiple classification algorithms were compared with
and without feature selection through Recursive Feature Elimination (RFE) including SVMs
(Support Vector Machines) with either Linear and Polynomial parameterizations, Bayesian
Logistic Regression (BLR) with either Gaussian or Laplace priors, and Random Forests. Each
progressive time point includes features from that time point and the previous time point(s).
Figure 2:
Graphical representation of the percentage of times each feature is selected across 5X10 fold
cross-validations through SVM Recursive Feature Elimination (RFE) based on clinical and chart
features alone.
Note: Analyses reflect the inclusion of additional time points. The blue line represents the 55%
cut-off for establishing stability.
Figure 3
Graphical representation of the percentage of times each feature is selected across 5X10 fold
cross-validations through SVM Recursive Feature Elimination (RFE) based on neuroendocrine
features alone.
Note: Analyses reflect the inclusion of additional time points. The blue line represents the 55%
cut-off for establishing stability.
Figure 4
Graphical representation of the percentage of times each feature is selected across 5X10 fold
cross-validations through SVM Recursive Feature Elimination (RFE) based on combined
neuroendocrine, clinical, and chart features.
.
Note: Analyses reflect the inclusion of additional time points. The blue line represents the 55%
cut-off for establishing stability.
1.
Goenjian AK, Yehuda R, Pynoos RS, Steinberg AM, Tashjian M, Yang RK, et al. (1996): Basal
cortisol, dexamethasone suppression of cortisol, and MHPG in adolescents after the 1988 earthquake in
Armenia. American Journal of Psychiatry. 153:929-934.
2.
Yehuda R, Teicher MH, Trestman RL, Levengood RA, Siever LJ (1996): Cortisol regulation in
posttraumatic stress disorder and major depression: a chronobiological analysis. Biological psychiatry.
40:79-88.
3.
Muthen LK, & Muthen, B.O. (1998): Mplus user's guide. 3 ed. Los Angeles: Muthen & Muthen.
4.
Nylund KL, Asparouhov T, Muthen BO (2007): Deciding on the number of classes in latent class
analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling.
14:535-569.
5.
Duncan TE, Duncan SC, Strycker LA (2006): An introduction to latent variable growth curve
modeling: Concepts, issues, and applications (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates
Publishers; US.
6.
Bradley AP (1997): The use of the area under the ROC curve in the evaluation of machine
learning algorithms. Pattern Recognition. 30:15.
7.
Fawcett T (2003): ROC Graphs: Notes and Practical Considerations for Researchers. Technical
Report, HPL-2003-4, HP Laboratories.
8.
Harrell F, Lee KL, Mark DB (1996): Tutorial in biostatistics multivariable prognostic models: issues
in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
Statistics in medicine. 15:361-387.