Dr Liam J. McGuffin
RCUK Academic Fellow
[email protected]
McGuffin Group Methods
for Quality Assessment
Three methods for different categories:
•
ModFOLD v 1.1 – Server, QMODE1
•
ModFOLDclust – Server, QMODE2
•
ModFOLD v 2.0 – Human, QMODE1
(now a server, QMODE2)
13 July 2017
© University of Reading 2007
www.reading.ac.uk/bioinf
ModFOLD v 1.1 (Server)
• Combines 6 QA scores using a Neural Network (4 scores
in CASP7)
• Considers models individually
• Trained using TM-scores and fold recognition models
• Outputs a single score for each model (QMODE1)
SS (new)
SS-weighted (new)
ModSSEA
MODCHECK
TM-score
ProQ-MX
ProQ-LG
To put your footer here go to View > Header and Footer
2
ModFOLDclust (Server)
• Simple clustering method - unsupervised
• Compares all sever models against one another
• Outputs overall score plus per-residue accuracy
(QMODE2)
1. Overall/global model quality Mean TM-score between models
(Similar to 3D-Jury)
1
S
Tm

N  1 mM
S = quality score for model
N-1 = number of pairwise structural alignments
carried out for model
M = set of alignments
Tm = TM-score for alignment of models
2. Per-residue accuracy Mean S-score rearranged to give
distance in Angstroms
Si 
1
2
d 
1   i 
 d0 
1
Sr 
Sia

N  1 aA
 1  
d r  d 0     1
  Sr  
To put your footer here go to View > Header and Footer
Si = S-score for residue i
di = distance between aligned residues
according to TM-score superposition
d0 = distance threshold (3.9)
Sr = predicted residue accuracy for the
model
N = number of models
A = set of alignments
Sia = Si score for a residue in a structural
alignment (a)
3
ModFOLD v 2.0 (Manual)
• Combines ModFOLD scores, ModFOLDclust score and initial
server ranking using a NN
• Considers models individually (sort of)
• Compares each model against 30 nFOLD3 server models to get
a ModFOLDclust score (server version)
• Per-residue accuracy from ModFOLDclust method (server
version)
Server rank (new)
ModFOLDclust (new)
SS (new)
SS-weighted (new)
TM-score
ModSSEA
MODCHECK
ProQ-MX
ProQ-LG
To put your footer here go to View > Header and Footer
4
ModFOLD 2.0 - all TS1 models
Predicted quality
Predicted quality
ModFOLDclust – all TS1 models
Observed quality (GDT-TS)
Observed quality (GDT-TS)
ModFOLDclust – T0499
Predicted quality
Predicted quality
ModFOLDclust – T0498
To put your footer
here
go to View > Header and Footer
Observed
quality
(GDT-TS)
Observed quality (GDT-TS)
5
Results continued…
Wilcoxon signed rank sum tests
Correlation of output with GDT-TS
Method
Kendall
(Tau)
Spearman
(Rho)
Pearson
(R)
ModFOLDclust
0.76
0.91
0.92
ModFOLD 2.0
0.74
0.90
0.91
ModFOLD 1.1
0.52
0.71
0.71
Conclusions
(H0 = GDTx ≤ GDTy, H1 = GDTx > GDTy)
ModFOLDclust
Zhang
-Server
ModFOLD
2.0
pro-sp3TASSER
ModFOLDclust
1.000
0.181
0.147
0.000
Zhang-Server
0.820
1.000
0.162
0.000
ModFOLD 2.0
0.854
0.839
1.000
0.000
pro-sp3TASSER
1.000
1.000
1.000
1.000
• ModFOLD 1.1:
• Increase in average per-target correlation since CASP7?
• Decrease in global correlation? But diff. data sets.
• ModFOLD 2.0:
• Fewer outliers but no significant difference from ModFOLDclust
• Benchmarking on CASP7 set showed an increase in Kendall’s Tau (not
significant, training artefact?)
• ModFOLDclust:
• Most simple & effective method, but CPU intensive
• Still room for improvement, doesn’t consistently recognise best model
• Marginally better than Zhang-Server in terms of cumulative GDT-TS, but
difference is not significant
To put your footer here go to View > Header and Footer
6
The ModFOLD server
Method
Relative
speed
Upload
options
Output
mode
ModFOLD 1.1
Fast
Single and
multiple
QMODE1
ModFOLDclust
Slow
Multiple
only
QMODE2
ModFOLD 2.0
Medium
Single and
multiple
QMODE2
http://www.reading.ac.uk/bioinf/ModFOLD/
[email protected]
References:
• McGuffin, L. J. (2008) The ModFOLD Server for the Quality Assessment of
Protein Structural Models. Bioinformatics, 24, 586-7.
• McGuffin, L. J. (2007) Benchmarking consensus model quality assessment for
protein fold recognition. BMC Bioinformatics, 8, 345.
To put your footer here go to View > Header and Footer
7