!"
%
'
)
(
#
#
&
$
*! + ,
*
-
+
+ .
(
!
%
% 0 1
(
% 0 1
/
+
+* ",+
! $ * + ! !'
(
2
3
"4 4
$
' *+ 5+
! "&
$
%
(
(
&
6
6 &
&
7
#
6
#
7 8
9 :+* !
;
7
, *+
<
4
%
!
,
,
*
*
!" $
(
%
=
7
(
=
!+* .
4
4
4
&
%
0 1
*
!
4
4
+
.
>%
+ %
(
&
4
? 0 1
@
%
&
%
,
A
";&
<(
0=
BCCD$
)
) 4
(
(
7
(
"
"
&
(
$
$
( " $ %&
!
+, *.
"
$
%&
62
6
'
"
!
E
6(
, 4
%&
=
1 0
FGGH$
6( %&
&
I
$(
"5 % =
"
!
%
& 4J
$(
0.0
0.2
0.4
Penalty
0.6
0.8
1.0
Power family
0.0
0.2
0.4
0.6
0.8
1.0
0.8
1.0
Coefficient absolute value
0.0
0.2
0.4
Penalty
0.6
0.8
1.0
Generalized elastic net
0.0
0.2
0.4
0.6
I
"B$
'
'
!
(
"F$
%
"D$
I
4
(
% 4
4
6
"B$
+
'
(
%
/ !'
4
1 4 %
(
4
E
4K
(
+
E (
4
K
*
$
"
, 4
8 (
=
L+, *
(
*
7
*+
"
1
+
'
)
&
$
7 %&
"
$
1
6
"
$
(
&
"
(
$
'
4
"B$
"F$
E
4
0
"'
$
(
6
"
6$
L+, *
1
E
"M$
+ 2, J
%
6
2
&
6
%
&
&
2 +
(
4
(
4
E (
*
%
"
$
,
6
'
4
4
% %
4
2
'
%
6
4
4 1
)4
(
&
(
4
)4
&
% % "
% % "&
& 1
$(
%
$
%
(
&
(
*+
N 6
2
"
"
FGGF$
6$2 &
0N'
Elastic Net 1.9
Elastic Net 1.5
3
9
400
0.0
0.1
0.2
0.3
0.4
200
7
0
5
2
4
10
8
6
1
−200
8
10
1
6
Coefficients
200
7
0
4
−200
Coefficients
400
3
9
5
2
0.5
0.0
0.1
0.2
0.3
R − squared
R − squared
Elastic Net 1.25
Lasso
0.4
0.5
3
9
400
0.0
0.1
0.2
0.3
R − squared
0.4
0.5
200
7
0
5
2
4
10
8
6
1
−200
10
8
6
1
Coefficients
200
7
0
4
−200
Coefficients
400
3
9
5
2
0.0
0.1
0.2
0.3
R − squared
0.4
0.5
0.1
0.2
0.3
0.4
500
0
8
10
1
7
2
0.5
5
0.0
0.1
0.2
0.3
Elastic Net 0.25
Elastic Net 0.0
−500
2
5
0.1
0.2
0.3
R − squared
0.4
0.5
3
500
5
6
8
9
7
1
4
2
0
0.5
9
9
3 999 9
9
3
3
3
9 6
3
33
6
6
6
4
4
4
44 4
88
8
110
5
5
6
6
6
61
8
8
8
88
7
7
7
77
71
1
4
21
21
211
7
27
55
2
22
−500
8
10
1
7
Coefficients
500
0
1000
R − squared
9
3
6
4
0.0
0.4
R − squared
1000
0.0
Coefficients
9
3
6
4
−500
0
4
7
10
8
6
1
5
2
Coefficients
500
3
9
−500
Coefficients
1000
Elastic Net 0.5
1000
Lasso
55
5
5
0.0
0.1
0.2
0.3
R − squared
0.4
0.5
*
#
%4+
4
2
%
N 6
"
6$2 &
2
%
0N'
Elastic Net 1.9
Elastic Net 1.5
7
0.05
0.10
0.15
0.4
0.2
7
0.20
0.00
0.05
0.10
0.15
Explained devience
Explained devience
Elastic Net 1.25
Lasso
1
8
4
0.6
9
0.4
5
2
3
6
1
8
4
7
0.00
0.05
0.10
0.15
Explained devience
0.20
0.20
0.2
0.2
0.4
5
2
3
6
Coefficients
0.6
9
0.0
Coefficients
1
4
8
0.0
0.00
5
2
3
6
0.0
8
Coefficients
0.2
0.0
1
4
−0.2
Coefficients
0.4
5
2
3
6
9
0.6
9
7
0.00
0.05
0.10
0.15
Explained devience
0.20
Lasso
Elastic Net 0.5
0.6
8
4
7
0.05
0.10
0.15
0.4
0.2
1
4
8
7
0.20
0.00
0.05
0.10
0.15
Explained devience
Explained devience
Elastic Net 0.25
Elastic Net 0.0
0.2
0.4
5
6
2
3
−0.2 0.0
1
4
Coefficients
0.6
9
0.20
0.2 0.4 0.6 0.8
0.00
5
6
2
3
0.0
0.0
1
9
5
6
2
3
1
4
8
8
7
0.00
0.05
0.10
0.15
Explained devience
0.20
−0.2
Coefficients
9
−0.2
Coefficients
0.4
5
2
3
6
0.2
Coefficients
0.6
9
7
0.00
0.05
0.10
0.15
Explained devience
0.20
( %
O
6
2
K
!
"
"
#
#
#
500
200
100
50
20
Non zero coefficients
10
5
2
1
0.0
0.2
0.4
0.6
0.8
R − squared
B C2 B P2 B H2 B G "
2 & % $2 G H2 G D2 G F2 G B2
1.0
*
#
%
O
K
!
7
$
$
6
500
100
Non zero coefficients
50
10
5
1
0.0
0.2
0.4
0.6
0.8
Explained devience
B C2 B P2 B H2 B G "
2 & % $2 G P2 G H2 G D2 G G
1.0
E
4
" $
( %
O
6
2
&%
, 4
;
<
%4
(
'
(
1
!
1
!
2
!
!
2
"B H$
" 0$
Corr = 0.4
50
100
Variables entered
0.6
0.4
0
0.0
0.2
Model error
150
0.8
200
1.0
Corr = 0.4
step
0.0
0.1
0.2
0.5
lasso
step
elnet 1.5
0.0
0.2
0.5
lasso
elnet 1.5
Corr = 0.4, a > 0
0.1
0.2
Model error
0.6
0.4
0.2
0.0
0.0
Model error
0.3
0.8
0.4
1.0
Corr = 0.0
0.1
step
0.0
0.1
0.2
0.5
lasso
elnet 1.5
step
0.0
0.1
0.2
0.5
lasso
elnet 1.5
Corr = 0.4
25
15
0.6
0
0.2
5
0.4
Model error
0.8
Corr = 0.4
0.2
bridge
lasso
0.0
0.1
lasso elnet 1.5
20
15
5
10
0.6
0
0.2
Model error
0.5
Corr = 0.0
1.0
Corr = 0.0
0.2
0.1
bridge
lasso
0.0
0.1
0.5
lasso elnet 1.5
Corr = 0.4, a > 0
30
0.20
10
20
0.15
0
0.10
Model error
40
Corr = 0.4, a > 0
0.2
elnet 1.5
bridge
lasso
0.0
0.1
0.2
0.5
lasso elnet 1.5
O
"B$
6"
$
"
"F$ %
&
"D$
$
%
%
"
&
$(
&
&%
4
0
4
%
Corr = 0.4, Post
Variables entered
0.6
0
0.0
0.2
50
0.4
Model error
100
0.8
150
1.0
Corr = 0.4, Post
step
0.0
0.1
0.2
0.5
lasso
elnet 1.5
step
0.0
0.2
0.5
lasso
elnet 1.5
lasso
elnet 1.5
Corr = 0.4, a > 0, Post
0.0
0.2
0.0
0.2
0.1
0.4
Model error
0.6
0.3
0.8
1.0
0.4
Corr = 0, Post
Model error
0.1
step
0.0
0.1
0.2
0.5
lasso
elnet 1.5
step
0.0
0.1
0.2
0.5
! *
"B$ &
"
!
$
7
"F$
&
4
"D$ 14
&
"H$
6
&
"Q$ &
&
= 4
4
%
&%
&
0
(
'
4
&
E
4
=
(
6
%
1 4(
4
6
6
2
% 4
,%
0
6
2
%
4
I
!
"
#
6$
(
R
%
BG7
6
(
S%
E
"
%
&
6$
1 4
!
"
$
+*/
4
(##111O
%#TA4 #
0 #'
%#TA4 #
#'
+
4
(##111O
0