4/9/2009
QM 6203: Econometrics I
Mohd‐Pisal Zainal, Ph.D
Department of Banking
INCEIF
Basic Econometrics
Chapter 8
MULTIPLE REGRESSION ANALYSIS: The Problem of Inference
1
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-3. Hypothesis testing in multiple regression:
•Testing hypotheses about an individual partial regression coefficient
•Testing the overall significance of the estimated multiple regression model, that is, finding out if all the partial slope coefficients are simultaneously equal to zero
•Testing that two or more coefficients are equal to one another
•Testing that the partial regression coefficients satisfy certain restrictions
•Testing the stability of the estimated regression model over time or in different cross‐sectional units
•Testing the functional form of regression models
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-4. Hypothesis testing about individual partial regression coefficients
With the assumption that u i ~ N(0,σ2) we can use t‐test to test a hypothesis about any individual partial regression coefficient.
H0: β2 = 0
H1: β
: β2 # 0
If the computed t value > critical t value at the chosen level of significance, we may reject the null hypothesis; otherwise, we may not reject it
2
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐5. Testing the overall significance of a multiple regression: The F‐Test
i
Th F T t
For Yi = β1 + β2X2i + β3X3i + ........+ βkXki + ui
• To test the hypothesis H0: β2 =β3 =....= βk= 0 (all slope coefficients are simultaneously zero) versus H1: Not at all slope coefficients are simultaneously zero, compute
•F=(ESS/df)/(RSS/df)=(ESS/(k‐1))/(RSS/(n‐k)) (8.5.7) (k = total number of parameters to be estimated including intercept)
• If F > F critical = Fα(k‐1,n‐k), reject H0
• Otherwise you do not reject it
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐5. Testing the overall significance of a multiple regression
z Alternatively, if the p‐value of F obtained from (8.5.7) is sufficiently low, one can reject H0
z An important relationship between R2 and F:
F=(ESS/(k‐1))/(RSS/(n‐k)) or
R2/(k‐1)
/(k 1)
F = ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
(1‐R2) / (n‐k) ( see prove on page 249)
(8.5.1)
3
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐5. Testing the overall significance of a multiple regression in terms of R
in terms of R2
For Yi = β1 + β2X2i + β3X3i + ........+ βkXki + ui
z To test the hypothesis H0: β2 = β3 = .....= βk = 0 (all slope coefficients are simultaneously zero) versus H1: Not at all slope coefficients are simultaneously zero, compute
z F = [R
F = [R2/(k‐1)] / [(1‐R
/(k‐1)] / [(1‐R2) / (n‐k)] (8.5.13) (k = total ) / (n‐k)] (8 5 13) (k = total
number of parameters to be estimated including intercept)
z If F > F critical = F α, (k‐1,n‐k), reject H0
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐5. Testing the overall significance of a multiple regression
zAlternatively, if the p‐value of F obtained from (8.5.13) is sufficiently low, one can reject H0
The “Incremental” or “Marginal” contribution of
an explanatory variable:
Let βX is the new ((additional)) term in the right
g
hand of a regression. Under the usual
assumption of the normality of ui and the HO: β
= 0, it can be shown that the following F ratio
will follow the F distribution with respectively
degree of freedom
4
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐5. Testing the overall significance of a multiple regression
[R2new ‐ R2old] / Df1
F com = ‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐
(8.5.18)
2
[1 ‐ R new] / Df2
Where Df1 = number of new regressors
Df2 = n – number of parameters in the new model
new model
R2new is standing for coefficient of determination of the new regression (by adding βX);
R2old is standing for coefficient of determination of the old regression (before adding βX).
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐5. Testing the overall significance of a multiple regression
g
Decision Rule:
If F com > F α, Df1 , Df2 one can reject the Ho that β = 0 and conclude that the addition of X to the model significantly increases ESS and hence the R2 value
f || f
ff
™ When to Add a New Variable? If |t| of coefficient of X > 1 (or F= t 2 of that variable exceeds 1)
™ When to Add a Group of Variables? If adding a group of variables to the model will give F value greater than 1;
5
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐6. Testing the equality of two regression coefficients
Yi = β1 + β2X2i + β3X3i + β4X4i + ui
(8.6.1)
Test the hypotheses:
(8.6.2)
H0: β3 = β4 or β3 - β4 = 0
H1: β3 # β4 or β3 - β4 # 0
Under the classical assumption it can be shown:
t = [(β^3 - β^4) – (β3 - β4)] / se(β^3 - β^4)
follows the t distribution with (n-4) df because (8.6.1) is a fourvariable model or
or, more generally,
generally with (n
(n-k)
k) df.
df where k is the
total number of parameters estimated, including intercept
term.
se(β^3 - β^4) = √ [var((β^3) + var( β^4) – 2cov(β^3, β^4)]
(see appendix)
(8.6.4)
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
t = (β^3 - β^4) / √ [var((β^3) + var( β^4) – 2cov(β^3, β^4)]
(8.6.5)
Steps for testing:
1. Estimate β^3 and β^4
2. Compute se(β^3 - β^4) through (8.6.4)
3. Obtain t‐ ratio from (8.6.5) with H0: β3 = β4
4. If t‐computed > t‐critical at designated level of significance for given df, then reject H0. Otherwise do not reject it. Alternatively, if the p‐value of t statistic from (8.6.5) is reasonable low, one can reject H0.
•
Example 8.2: The cubic cost function revisited
6
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐7. Restricted least square: Testing linear
equality restrictions
Yi = β1X β22i X β33i eui (7.10.1) and (8.7.1)
Y = output
X2 = labor input
X3 = capital input
In the log‐form:
l Yi = β
lnY
β0 + β
+ β2lnX
l X2i + β
+ β3lnX
l X3i + u
+ i (8.7.2)
(8 7 2)
with the constant return to scale: β2 + β3 = 1 (8.7.3)
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐7. Restricted least square: Testing linear
equality restrictions
How to test (8.7.3)
z The t Test approach (unrestricted): test of the hypothesis H0: β2 + β3 = 1 can be conducted by t‐ test:
t = [(β^2 + β^3) – (β2 + β3)] / se(β^2 - β^3)
(8.7.4)
z The F Test approach (restricted least square ‐RLS): Using, say, β2 = 1‐β3 and substitute it into (8.7.2) we get: ln(Yi /X2i) = β0 + β3 ln(X3i /X2i) + ui (8.7.8). Where (Yi /X2i) is output/labor ratio, and (X3i / X2i) is capital/labor ratio
7
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐7. Restricted least square: Testing linear equality restrictions
Σu^2UR=RSSUR of unrestricted regression (8.7.2) of unrestricted regression (8 7 2)
2
and Σ u^ R = RSSR of restricted regression (8.7.7), m = number of linear restrictions,
k = number of parameters in the unrestricted regression,
n = number of observations.
2
2
R UR and R R are R2 values obtained from unrestricted and restricted regressions respectively. Then
F=[(RSSR – RSSUR)/m]/[RSSUR/(n‐k)] = = [(R2UR – R2R) / m] / [1 – R2UR / (n‐k)] (8.7.10)
follows F distribution with m, (n‐k) df.
Decision rule: If F > F m, n‐k , reject H0: β2 + β3 = 1
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐7. Restricted least square: Testing linear equality restrictions
2
•
Note: R UR ≥ R2R
(8.7.11)
2
2
(8.7.12)
•
and Σ u^ UR ≤ Σ u^ R
z Example 8.3: The Cobb‐Douglas Production function for Taiwanese Agricultural Sector, 1958‐1972. (pages 259‐260). Data in Table 7.3 (p g
(page 216)
)
z General F Testing (page 260)
z Example 8.4: The demand for chicken in the US, 1960‐1982. Data in exercise 7.23 (page 228)
8
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐8. Comparing two regressions: Testing for structural y
g
stability of regression models
Table 8.8: Personal savings and income data, UK, 1946‐
1963 (millions of pounds)
Savings function:
z Reconstruction period:
Y t = α1+ α2X t + U1t (t = 1,2,...,n1)
z Post‐Reconstruction period:
Y t = β1 + β2X t + U2t (t = 1,2,...,n2)
Where Y is personal savings, X is personal income, the us
are disturbance terms in the two equations and n1, n2 are the number of observations in the two period Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐8. Comparing two regressions: Testing for structural stability of regression models
regression models
+ The structural change may mean that the two intercept are different, or the two slopes are different, or both are different, or any other suitable combination of the parameters. If there is no structural change we can combine all the n1, n2 and just estimate one savings function as:
Y t = λ1 + λ2X t + Ut (t = 1,2,...,n1, 1,....n2). (8.8.3)
How do we find out whether there is a structural change in the savings‐income relationship between the two period? A popular test sa
ings income relationship bet een the t o period? A pop lar test
is Chow‐Test, it is simply the F Test discussed earlier
HO: αi = βi ∀i Vs H1: ∃i that αi # βi
9
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐8. Comparing two regressions: Testing for structural stability of regression models
regression models
+ The assumptions underlying the Chow test
u1t and u2t ~ N(0,s2), two error terms are normally distributed
with the same variance
u1t and u2t are independently distributed
Step 1: Estimate (8.8.3), get RSS, say, S1 with df = (n1+n2 – k);
k is number of parameters estimated )
Step 2: Estimate (8.8.1)
(8 8 1) and (8.8.2)
(8 8 2) individually and get their
RSS, say, S2 and S3 , with df = (n1 – k) and (n2-k) respectively.
Call S4 = S2+S3; with df = (n1+n2 – 2k)
Step 3: S5 = S1 – S4;
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8‐8. Comparing two regressions: Testing for structural stability of regression models
Step 4: Given the assumptions of the Chow Test, it can be
show that
(8.8.4)
F = [S5 / k] / [S4 / (n1+n2 – 2k)]
follows the F distribution with Df = (k, n1+n2 – 2k)
Decision Rule: If F computed by (8.8.4) > F- critical at the
chosen level of significance a => reject the hypothesis that the
regression (8.8.1) and (8.8.2) are the same, or reject the
hypothesis of structural stability; One can use p-value of the F
obtained from (8.8.4) to reject H0 if p-value low reasonably.
+ Apply for the data in Table 8.8
10
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-9. Testing the functional form of
regression:
Choosing between linear and log‐linear regression models: MWD Test (MacKinnon, White and Davidson)
H0: Linear Model : Linear Model Y is a linear function of Y is a linear function of
regressors, the Xs;
H1: Log‐linear Model Y is a linear function of logs of regressors, the lnXs;
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
8-9. Testing the functional form of regression:
Step 1: Estimate the linear model and obtain the
estimated Y values. Call them Yf (i.e.,Y^). Take lnYf.
Step 2: Estimate the log-linear model and obtain the
estimated lnY values, call them lnf (i.e., ln^Y )
Step 3: Obtain Z1 = (lnYf – lnf)
Step 4: Regress Y on Xs and Z1. Reject H0 if the
coefficient of Z1 is statistically significant, by the usual t test
Step 5: Obtain Z2 = antilog of (lnf – Yf)
Step 6: Regress lnY on lnXs and Z2. Reject H1 if the
coefficient of Z2 is statistically significant, by the usual ttest
11
4/9/2009
Chapter 8
MULTIPLE REGRESSION ANALYSIS:
The Problem of Inference
Example
p 8.5: The demand for Roses (p
(page
g 266267). Data in exercise 7.20 (page 225)
8-10. Prediction with multiple regression
Follow the section 5-10 and the illustration in pages
267-268 by using data set in the Table 8.1 (page
241)
8 11 The troika of hypothesis tests: The likelihood
8-11.
ratio (LR), Wald (W) and Lagarange Multiplier (LM)
Tests
8‐12. Summary and Conclusions
12