July 10, 1984 Autoregressive Conditionally ~eteroscedastic Models Sastry G. Pantula Department of Statistics North Carolina State University Raleigh, NC 27695-8203 Key Words and Phrases: Autoregression; heteroscedasticity; maximum likelihood and generalized least squares estimators; asymptotic normality. Abstract Two linear regression models, where the independent variables are (4) fixed and bounded, and (b) the lagged values of the dependent variable, with autoregressive conditionally heteroscedastic (ARCH) errors are considered. A series representation and some ergodic properties of the first order ARCH errors are derived. The strong consistency and the asymptotic normality of the maximum likelihood estimators are established. Asymptotic distributions of the least squares estimator and an estimated generalized least squares estimator are also derived. Autoregressive Conditionally Heteroscedastic Models Sastry G. Pantula North Carolina State University 1. Introduction: Consider a process {Y } satisfying the equation t (1.1) Yt • X a + e ~t~ t where X : 1 x p is a known row vector and ~: p ~t parameters. X 1 is a vector of unknown Traditional econometric models assume that e t is a sequence of 2 uncorrelated (0,0 ) random variables and the conditional mean of Y give~ the t past is !t~' where !t may contain lagged values of Yt • function, the best forecast of Y , based on the past t Under a symmetric loss in~ormation, is the conditional mean of Y given the past and is denoted by E[YtIFt_lJ, where t F - • information up to time (t-l). t l Therefore for the model (1.1), the one step forecast error is The unconditional variance of the one period forecast is given by 0 2 . Note that and For the conventional econometric models, however, the conditional variance does not depend on F t- 1. For some processes one might expect better forecast intervals if additional information from the past were allowed to affect the forecast variance. A model which allows the conditional variance to depend on the past -2- realization is the bilinear model described by Granger and Anderson (1978). Jones (1965), Granger and Anderson (1978), and Priestly (1978) considered nonlinear time series models. Engle (1982) proposed a class of models, called autoregressive conditionally heteroscedastic (ARCH) models, for which both the conditional mean and the . conditional variance of a time series are functions of the past behavior of the time series. If the conditional density is normal, then a general expression for the ARCH model is where g and h are measurable. A special case we consider assumes that the mean can be expressed as a linear combination of variables in the information set, while the variance is a qth order weighted average of the squares of past di.sturbances. More precisely, where and e Assume that t = Y - X a t -t .. aI' a2 , •.. , aq are nonnegative and aO is positive. Although Yt is conditionally normal, Engle (1982) established that the Yt are not jointly normal and the marginal distribution of Y is not normal. t restrict to the case q = 1. In this manuscript w" -3- Engle (1982) derived the moments of the let} process for q = 1. In section 2 we derive a representation for the let} process and use it to derive the moments of the let} process. let} process. We also derive the ergodic properties of the Engle (1982) also considered the maximum likelihood estimation of the parameters. He established that the information matrix is block diagonal, . indicating that the maximum likelihood estimators of ~ and~are independent. He also indicated that the maximum likelihood estimators are asymptotically normal. In section 3 we formally derive the asymptotic distribution of the maximum likelihood estimators. We also derive the asymptotic properties of the least squares and estimated generalized least squares estimators of 2. ~ and ~. Properties of the ARCH Models The simplest ARCH model is the first order linear model given by y t =X a -t- (2.1) where (2.2) and Ft - l = a-field generated by Y and e ' sSt-I, t • 1, 2, .•. s s It is assumed that So is strictly positive and 8 In the 1 is nonnegative. following theorem we obtain a representation for the {e~} sequence which is useful in deriving the properties of the model (2.1). Theorem 2.1: Let let} be a sequence of random variables satisfying (2.2). Assume that So >0 and 0 ~ 81 < 1. It is assumed that the process began indefinitely in the past with a finite initial variance. e where {Zt; variables. t 2 t ~E Bt ( tw = So t=O 1 i=O Z2 .) t-~ a.s. , Then, (2.3) = 0, ±l, ±2, ... } is a sequence of normal independent (0,1) -4- Proof: Since e is conditionally normal, we get t where Zt is a standard normal random variable independent of e - l . t e Therefore, 2 = Z2( 13 2 t 0 + Sle t _ l ) t 2 2 2 = Zt [SO + SlZt_l(SO + 6l e t _ 2 )] j-l l l 2 j j-l 2 2 I: 6 ( 'IT Zt-i) + 6 ( 'IT Zt-i)e = 6 0 l=O 1 i=O 1 i=O t j Note that, l Z2 .)] = (1 - 1>1 " )-1 < (i'IT l=o 1 =O t-l. E[ 00 I: 6l 00. Therefore, by result (xi) of Chung (1974, p. 42), co I: l=O l l 2 13 ('IT Z ) < 1 i=O t-i a.s . . co Also, assuming that the process was initiated in the infinite past with a finite initial variance, co E [ I: j=O Using the same argument, co "j (j; 1 I: j=O 1>1 i=O 2 Z . )e t-1. 2 < co . t-J a. s. , and j j -1 Z2 6 ( 11' 1 i=O ) 2 t-i e t-j - 0 a •s ., as j + co • Therefore, 2 e t = 60 co I: l=O l l Z2 .) 61 ('IT t-l. i=O A similar expression for e 2 t a.s. 0 can be obtained in case q > 1. Note that e t has a sYmmetric distribution and that the sequence {e } is strictly t stationary. The probability density function f of e t can be obtained by solving -5x ~ f f(x) • 2 2(SO+B u 2 ) l e -00 We have not been able to obtain a solution for (2.4). obtain the moments of the {e } process. t Theorem 2.2: We use (2.3) to See also Engle (1982). Let {e } be a sequence of random variables satisfying the t conditions of Theorem 2.1. m 2r Then, = E[e 2t r ] < 00 i f and only i f er • Br1 Also, if e r r ~ (2j-l) j=l <1 • < 1, then r-j Proof: Note that, . and by Monotone convergence theorem (see Chung, 1974, p. 42), Using Minkouski's inequality, E[e~r] :a 13 0r .tim f . n~ n-- . J=O • a0r < Also, 00 .tim n-if [n. ~ J=O j B1 {E 13 er < 1 j i~O Z 2r 1 } r t-i -1 e;j+l)/rr 1 . r ] -6- E[e;r] Therefore, M Zr sjr E( 'II'j zZr ) 0 j=O 1 i=O t-i co -j = Sr0 r Sl 6( j+l) r j=O ~ Sr = co co r if 6 is finite iff 6 r r ~ 1 < l. Now, = E[e;r] M Zr .. E[ZZr<S + Sl e2 )r] t 0 t-1 r r-j sj -r = r (~) m 6 Sl So J 1 Zj r j=O r-l ,. 6 (1-6 )-1 r (~) r r j=O J Note that if 3S 2 1 < 1, So Sl r-j m 2j 0 then v(e~) .. 204(1-3Si)-1 and where 2 is a sequence uncorrelated (0,0 ) random variables. We now give two lemmas that are useful in obtaining the ergodic properties of the {e } process. t Lemma 2.1: F n ~ F + . n l Let {U } be a sequence of F measurable random variables and n n Suppose there exists an integrable random variable U and a constant c such that p[lu I >xl ;:icp[lul >xl. n Then -7- n -1 n k=l[U k - E(UkIFk_l)1 <~ If E[IUllog+lul1 -E,. .0. or if {Uk,k ~ I} and {E(UkIFk_l),k ~ I} are strictly stationary sequences then the covergence is almost sure. Proof: See Hall and Heyde (1980, p. 36). . Lemma 2.2: 0 A covariance stationary process {X } obeys the mean law of large t numbers, i.e., X n =n -1 n 1: t=l X t converges in mean square to a square integrable random variable X. process is st~ictly If the stationary and integrable, then X converges almost surely n to an integrable random variable. Proof: See Reve~sz 0 (1968, p. 99). In the following theorem we obtain the ergodic properties of {e } • t Theorem 2.3: er < 1, Consider {e } satisfying the conditions of Theorem 2.1. t then n n If 38~ < -1 a.s. , -1 n 1: tal e 2r-l -0 t a. s. . 1, then for a fixed j ; 0, n -1 n 1: tal e e .--+ t t-J 0 a.s. and a. s. . Proof: From Lemma 2.1 and Lemma 2.2, If -8-1 n E e 2t - M 2 n (say) , a.s. tal and a. s. , where a.s • . Therefore, a. s . . Also, - a. s. , and hence or .. S .. E[e~] • -1 n 2 M 2 0 S (1 - 1 )-1 Therefore n For r ~ a.s. 2, we use induction procedure to prove the result. n for s .. 1, 2, -1 n I' -1 n - Assume that a. s. , E tOIl ... , n and E tal e t - 1. 2r E tOIl e t - We know from Lemma 1 and Lemma 2 that a.s. (say), -9- Note that E[e 2rl F _ ]" t t 1 er r r (j) J'''0 cor; _ B1 2. e J t-1 Therefore, M 2r .. er r-1 r j=O (~) J c:~) or MOle (1- e )-1 r;:l 2r r r j-O r-j 2j + M er M 2r (~)(60)r-j M . 2J 61 J a.s. a.s. _ E[e 2r ] t Therefore, a. s. . Note that, if er < 1, then a.s. and hence n -1 n 2r-1 r e -- 0 tOIl t a.s. as n-+ ~. Similarly, and we g'!t n -1 n r tOIl et et - . J 0 a.s. as n- ~. Consider, -- 6 .. 0 2 4 E(e _ ) + 6 E(e _ ), 1 t 1 t 1 E(e~ e~_l)' Therefore, by Lemma 1, n -1 n 2 r e t e 2t-1-tOIl a. s. . a.s . -10- Similarly, n as n -1 n 2 2 1.: e e t=l t t-j 2 2 E(e t et_Jo) a.s. + .... In the next section we consider estimation of the parameters a and 3. §. Estimation of ARCH Models Given Y ' Y ' ... , Y satisfying (2.U, we desire to estimate ~ and § = (6 0 ,6 1 )'. n l 2 Engle (1982) used the method of scoring to obtain the maximum likelihood estimates, but did not formally derive the asymptotic properties of the maximum likelihood estimators. Engle (1982) indicated that if the conditions of Crowder (1976) are satisfied then the maximum likelihood estimates are asymptotically normal. We derive the limiting distribution of the maximum likelihood estimators by verifying the conditions of Hall and Heyde (1980, p. 174). We also consider an estimated generalized least squares estimator and derive its limiting distribution. We consider two particular choices for (a) ~t ~t is fixed and bounded; and For case (b), we also assume that ~l consider the likelihood given ~l' a (YO' Y- l , ••• , Y-P+l) is given and we We will also indicate how the results may be extended for the case where X consists of both fixed and lagged values and -t also for the case with X -t 3.1. that are not necessarily bounded. Maximum Likelihood Estimation Consider the log likelihood conditional on L (y) n - = -n -1 t=l ~ [in f (y t t IF t- l~:; ~l ' -11- where f (y t t IF t- 1) X' .. (~', ~' ) . is the conditional density of Y given the past and t From (2.2), (3.1) where Therefore, L (y) = constant n _ + (2n)-1 ~ bih \+ (2n)-1 ~ h-l(Y - tal \ ;I t=l t t X a)2 -t- (3.2) The maximum likelihood estimator in of X is the value of X that minimizes Ln(X)' Let X~ a (a~,~') be the true value of X. All probabilities and expectations are taken with respect to the true value XO. We assume that and (3.3) so that E[e 12 ] t <m (If X is fixed and bounded then we may assume that t 8 o S 6 S 0.3, so that E[e ] < m.) We also assume that ~O is in the interior 1 t of a compact set L. Therefore Xo is Let as~umed to be in the interior of f. r. defined below will be taken to be contained in All neighborhoods For 0 > 0, and IIx - xoll we obtain from the Taylor series expansion that Ln(r) • Ln(rO) + (r - ro)' ~:~~ + ~(y - y ~ ~o )' H (y-y ) n - ~o (3.4) < 0, -12- where and y* is a point between y and y o (not necessarily the same at each occurrence). We include a result from Hall and Heyde (1980) that we use to obtain the asymptotic properties of y -n Theorem 3.1: • Suppose that lim s~ n-+a> <5 * <5 -1/ T(y)I n .. J.] < co a.s. , 1 S i S P + 2 1 S j ~ p + 2 (3.5) a. s. , (3.6) and (3.7) . where ~ is a positive definite matrix of constants. of estimators fin} such that in converges to there is an event E with P(E) >1 - € Yo Then, there exists a sequence almost surely, and for and nO such that on E, for n € >0 > nO' Yn satisfies and L (y) attains a relative minimum at in • n - n -~ (aaLynj - L N(Q,~) y""yO where W is a positive definite matrix, then If, in addition, (3.8) -13- n Proof: ~(- rn rO ) -L - N(O _'!! -1 ~!! -1) . 0 See Hal! and Heyde (1980, p. 174). We now compute the partial derivatives of Ln(r). Note that, (y aL -aB n t-1 - X a)X' -t-1- -t-1 ' . --12n 1 n .. - r X' X t-1-t-1 n t=l (y n 1 - 2B 1 -n r t=l [(y t X )2 X' X t-! - -t-!~ -t-1-t-! - Xta) - - ] (y + X' X] - X a) [X'X t-l -t-l-t-t-1 -t-l-t h2 t 2 n [2(Y t -X-t-a) - h t ] 1 =- r 2n t:l1 a~a~' h3 t a2L n [(Yt-1 ~ -X a) 2 t-l -t-l- 4] (Yt-l-!t-l~) (y X a)2 -t-l- and a2 Ln a§a~' ¥ [(Yt-!t~)2- = -n1 t:ll ht ] h2 t [~] (y t-l - X a)X -t-l- -t-l 2 n [2(Y t -X-t-a) - h t ] r n t=l h2 t 1 [(Yt - 1- X a)] ;t-l:)3 (y t-l -t-!- X -t-1 X -t- l · -14- Recall that 1 e t = Zt v~t t = eO where v 2 + e l e t _l e ' ,. (eO,e ) is the true value of ~' = (SO,Sl) and Zt is a sequence of l independent N(O,l) variables. Therefore, 1 nE -~ + _1 e nE (Z~-l) e .~ Z X' X' tool-tool nt-I v t toot n 1 tal = and 1 n - - 2n tal E In the following theorem we verify the conditions (3.5) - (3.8) for the case (a) where we assume that X is fixed and bounded. -t Theorem 3.2: n -1 where ~ Assume that X -t ¥ is fixed and bounded. X'X __ A , tal -t .. t .. as n - "" , is a positive definite matrix. Then, the conditions (3.5) - (3.8) are satisfied with H - W,. [!!ll Q Q]., !!22 where !!U !!22 c ,. A(c .. ,. . ~ 2 1 + 2e l c 3 ) r c2 .. c 3 c3 ] c4 -1 c2 - E[v 2 ] -2 - E[v 2 ] c = E[etvi 2] = l 3 Also, assume that eil(c l - eOc 2) , -15- and c Proof: 4 = For a fixed i, let U n =v -~ n Z -1 n M X ., n, 1 where M" sup {X .}. i,n n,l Then, E[U n IFn- 1] = 0 2 and E(U ) is finite. n n -1 n ~ t=l U - t a.s. Therefore, by Lemma 2.1, 0 a.s. , as n~ = . Similarly, a.s. for k Also, and = 0, 1, 2, 3, 4 and t,s = 0,1 . -16- Note also that, =0 a.s. and Therefore, lim n n~ -1 n -1 = 0~ k v e - X t=l t t l -t-l a. s. , and a.s Now consider, coV(V~l, v~:j) = E[v~lv~:j] {E[v~1]}2 _ -1 { j-l l l 2 }-l v-I] :a E[v t ] E[ 6 0 + 6 1 So l;O 6 1 i~O Zt-l-i - t ~ 6- 3 6 j + l E( 2 .. 0 1 e t - l _j ) Therefore, as n - "" and since v -1 is bounded the convergence is almost sure. t Similarly, lim n n~ -1 n k t=l V -IX' t -t X = A c ~t l and lim n n~ Therefore, -1 n -2 -2 k v = E[V ] t t=l t a.s. a. s. , -17- a. s. , and Since and we observe that is positive definite. ~ Now we establish (3.5). Note that IT n (x)I 1J .. is a linear combination of terms of the form = h -t k ft(y; a, b k) -' = 0, where a 1, 2; b II X - Cons ider for = 0, Xo II < (Y t - X a)b - X a)a(y -tt-l -t-l- t - where Y* = A.y. + (l-A.)Y i 1 1 i, 1 2, 3. 15 , Ift(y;a, b, k) - f (yo;a, b, k)1 ~ 15 - = 1, 1, 2, 3, 4; and k 0 and 0 p+2 r i-l ~ Note that, - y.1, 01 Iy~ 1 ~ A.1 Iy.1 - y.1, 0 1 < 15 'X* < gt(a,b,k) , • We will show that af I for all X* and n -1 n r t=l ay: , lit - xoll < 15 , gt(a,b,k) converges almost surely to a finite limit. -18- For example, consider Note that, .. - 3h~4(Yt - Oft aS l 2 a) ?St~) (Y t - l - X -t-l- 6 Now, .. e Y - X -tooa t t - at where a t .. X (a - a ) • -t - -0 Then, -1 n 8 2 6 n L ete Note that, since we assumed that E[e ] is finite, t _l t t=l -1 n -1 n 6 2 L e - converge almost surely to finite constants. n L e and n t t=l t 1 t-1 other terms of IT (x)I .. can be bounded to establish (3.5). n 1.J , Now, using Scott's martingale central limit theorem (see Scott (1973» we Similarly, 1 (aL~ will show that n~ ~ is asymptotically normal. aX y"y - -0 Define, x-xo where D' = (Db, Sn" 11 1 , 11 ) is an arbitrary column vector such that 2 ~ a1 t~l -1(Z2 1) vt tOO 'X' ~ -~ Z n'X' et-lDO-t-l - t~l v t t~O-t D'n ~ O. Note that, -19- is a martingale. It is clear that {S ,F } n n Let ] V2 _ E[S2/ F n n n-l and s . 2 _ E[S2] n n Then, V2 - 2 n +~ n -1 -2 2 v t e - n'X' X n !)O + E v t n'X'X l -O-t-t-O -O-t-l-t-l t t-l t-l n e12 E n E t=l 2 -2 v t (n l + n 2e t _ l ) 2 and s 2 2 n n - 26 c E n' X' X n + c n' E X' X nO n 1 3 tal -O-t-l-t-l -0 1 -0 t-l -t-t - Note that, P as n - -1 co and o ~im n n-- -1 2 s n Since we assumed that Elv- 3 / 2 t z3 1 t , D~ D . E[e~] < co, < it follows that co ' and the Lindeberg condition is satisfied. s and -1 L S n n N(O,l) Therefore, -20- o Now we consider the case ~t = (Y t - l , Yt - 2 , .•• , Yt - p ), We assume that the roots of the characteristic equation p-l mP - a m 1 - a - lie inside the unit circle. p = 0 Then, Y can be written as an infinite moving t average as, co 1: j=O (3.10) w.e . J t-J where {w.l satisfy J =1 , j • 0 = 0 , j <0 In the following theorem we obtain the asymptotic properties of the maximum likelihood estimator. -Theorem 3.3: Assume that X -t = (y t- l' Yt- 2' •.. , Yt-p ) and that the roots of the equation (3.9) lie inside the unit circle. are satisfied with H = W= - 2) G -~l !!22 0 where and ~22 is as defined in Theorem 3.2. Proof: For a fixed i, let U n = v n-1 e n Y n-i Then the conditions (3.5) - (3.8) -21- Then, 2 and E[U ] is finite. Therefore, n n -1 n I: t=l a.s. U - 0 -2 2 (e -v ) e t "Also, n -1 n I: t=l v t t t t- lX 1 . t- ,4 -0 a.s. , and as n + 00 Using the arguments similar to those in Theorem 3.2, we get a. s. , = 0 = !!22 a.s . . = n Also, r =y-0 tim n+OO -1 n I: t=l n + tim 26 12 n -1 I: v -2 e 2 X' X n+OO t=l t t-l-t-l-t-l Consider, for fixed i and j n -1 n -1 I: v Y .Y . = t-4 t-J t=l t Now for fixed q and r, consider n -1 n I: t=l -1 vee t t-q t-r 00 I: k=O 00 w w n s=o k s I: -1 n I: t=l v -1 t e t - i - k e t-j-s -22I f q=r=l, then n -1 n v -1 2 tOIl t e t - 1 1: = (n6 ) 1 -1 n 1: tOIl -E[v -1 e 2 _ J t t 1 v -1 t (v -6 ) t 0 a.s. .If q"r>l, then it can be shown that for j > q. Therefore, a.s. Now if q ~ -1 r, then v e e are uncorrelated and hence t t-q t-r e -1 e .. E[V e e J t-q t-r t t -q t -r .. 0 a.s. Using Lemma 6.3.1 of Fuller (1976), we get lim n n~ -1 n -1 -1 1: v Y .Y . " E[V Y .Yt.J t t -1 -J tOIl t t-1 t-J a.s . . Similarly, lim n n~ and hence Note that -1 n -2 -2 1: v Y Y . " E[V Y 'Y .J t t -1 t -J tOIl t t-i t-J a.s. , -23- for any arbitrary ~O Now. E[v • -1 vp+1(Y p ' Yp - l ' .... Yl )· -1 , !t!t 1 is the variance covariance matrix of t If then or ~o !~l • 0 a.s . . However. we know that the variance covariance matrix of !p+l = (Y p .Y p- l •..•• Y1 ) is positive definite. Therefore. ~tl is positive definite. Now to establish (3.5). note that IT (Y)l , is a linear combination of terms -n - i J of the form -k a X a)b ql q2 q3 q4 ft(v;a,b.k.a) • h t (Y t - Xta) Xt -1, i Xt-.J l ' Xt.r Xt.~0 ~ ~ - - (y t- 1 - -t- 1 where qi =0 or 1 and a. band k range from 0 to 4. Again, for example. consider f (y_;2.4.3.0_) t = h-t 3 (y t - X a)2(y - X a)4 -tt-1 -t-1- Then :ii constant [e t 6 2 2 6 2 06 p2 y2 . e 6 _ + Y 1 .e + 0 e t t 1 i=l t- -1 t i=l t-1 t 1 + and 0 8 p 2 t ~ t i=l j=l y2 y2 . ] t-i t-J-l -24- 3 a t- 1) y t-1.. 11 ·Since E[e~2] is assumed to be finite, we get lim sup 0n~ 1 0+0 IT-n (x*)I l.J .. < Now we verify (3.8). ~ a.s. Let S = n' (aLaX ') n where n\ _ n' = (~~, n1 , n2 ) is an arbitrary vector of constants such that n'n ; Then, n r _!.< v"Z n'X' t=l t t_O-t vn2 = E[S2n IFn- 1] = 2 8 and Note that 2 ~ w 1 t=l v -2 e 2 n'X' X n + t t-l.0·t-l·t-l.0 v -1 n'X'X n t .O-t·t.O °. -25- as n - 1 , Since we assumed E[el~J t < 00 00 , and Therefore, by Scott's martingale central limit theorem, s -1 n S n L N(O,l) , and hence - YO) - L N(Q,~ -1 ) It is easy to see that if ~t o = (l,Y t - l , Yt - 2 , ... , Yt-p+l) the maximum likelihood estimator is still consistent and asymptotically normal. is fixed but not necessarily bounded then than n -~ . ~ may converge to a For example, if X = t, then (a - a ) t O X -t at a rate faster is 0p(n -1 ). Now we consider the least squares estimation of 3.2. O If y. Least Squares Estimation: The maximum likelihood estimates considered in 3.1 do not have explicit expressions and are estimated using iterative procedures. We now consider the ordinary and estimated generalized least squares estimates of y. The least squares estimates are obtained as follows: Step 1: of ~. Regress Y on X to obtain the ordinary least squares estimator a -t -t - -26- -2 -2 to get -6 and 6 , Let Regress e on a column of ones and e 0 1 t l t -2 v 6 + 6 e t 0 1 t l --1 -2 --1 --1 -2 e on v and v e - to get an estimated generalized Step 3: Regress v t t t t t l -2 + 6 e _ least squares estimates and • Let v .. l t l l t Step 4: Regress v-~ Y on v-~ X to get an estimated generalized least squares t t t -t Step 2: - - .. . eO estimate a - of e -t - Let {!t} and {Y } satisfy the conditions of Theorem 3.2. t YO be in the interior of f. ~ - (r - YO) - L Then N(2, ~O) and where y'" (~',§') 2 -1 B .. -1 o - cr A B .. o We first consider the case is fixed and bounded. Theorem 3.4: n A a We now study the properties of ~, ~, ~ and~. where X eO [ o -1 2 ~l -1] , ~l ~l Let -27- -1 - o -1 ] 2 !!22 ~l !!22 A .. E -1 and c l and !!22 as defined in Theorem 3.2. Proof: Note that ... , x' ) -n We know that Consider, S n .. n' _0 X'e n = tOIl 1: bte t where b t .. ~ n. i-l~' 0 X . t,~ and ~O is an arbitrary vector of constants with a martingale with, v2 .. E[S2 IF n n n- 1] a. s. , nb nO ~ O. Note that {S ,F } is n n -28- and s 2 n 2 '" (J Therefore, s L -2 2 V n n 1 and -1 n , 2 !lO sn - 2 ~!lO (J Since E[e:l is finite, the Lindeberg condition is satisfied and Therefore, ~ L 2 -1 n (':-':O)-N(Q,(J~ ). Now consider, n _ =[ ~ E t=2 (n-1) n E n -2 Ee t=2 t-1 t=2 where e and Note that, t = Y t - - X a -t- -29n -1 n -2 r et-la t - l + n t=2 r a t-l t=2 Similarly, n -1 n -4 t=2 t-l r = e and n -1 n -2 -2 r e e t=2 t-l t n = n -1 t=2 t 2 2 0 (n-~) et_le t + P Therefore, - n~ (a-a) - -- n 1 n [ -1 2 nEe t-2 t n -1 n r e2 t-2 t -1 n 4 r e t=2 t r [n-~ n r ta 2 dt ] n n -~ r dte~_l t=2 + o p (n-~) where d t - (Z2 - l)v t t Note that, n -1 n r t=2 e 2 t -1 n 4 n r e t=2 t as n + 00 r - A -I a. s. , • Con!3 ider , where nl and n2 are two arbitrary constants. Then S n is a martingale with -30- and Using the usual arguments we get s -1 S - L n N(O,1) , n and hence ~ L -1-1 n (~-~)--+ N(Q, ~l ~l ~l ) • Z Note that (Zt-l) and e t are uncorrelated and hence ~ and e are asymptotically Now we consider ~ obtained by step 3. independent. Using arguments as above, we get n n -e -1 v--2 e -2 ] t l t taZ n --Z -4 1: v t=Z t e t - l v- -2 2 t v--2 -2 t-Z t e t - l [ti = 1: and t=2 -1 + 0 p (1) n 1: t=Z Consider, n is a martingale with v 2 .. Z n n 1: (n t=Z Z Z n e _ ) 1 + Z t 1 and s Z n 1: v e --1 -2 ] t=Z t t n --1 -Z -2 1: v e e t=Z t t-l t n 1: !!Z2 Then, S [n = Z(n-l) E(n l 2 Z + n e _ ) Z t 1 a. s. , • -31- Since E[e~] is finite, Sn satisfies the Lindeberg condition and -1 L s n Sn - N(O,l) Therefore, A-l Now we consider the regression of v t A-l Y on v X t -t t where Let Then, as n -+ ... , and Consider, = ¥( ~ n X .)v·~ Z t=l i=l i,O t,1 t t Note again that S n is a martingale with and The Lindeberg condition is clearly satisfied and hence Note that, -32- s-IS n ~N(O,l) n and 2 Since (Z t -1) and Z t are uncorrelated . a . and 8 are asymptotically independent. .. A 0 Note that and the equality hold only.if 8 1 = O. Therefore, the maximum likelihood estimator is asymptotically the best among the three estimators considered and the estimated generalized least squares estimator is asymptotically better than the ordinary least squares estimator. Here we have assumed that X is fixed and bounded. .. t Suppose X is fixed .. t and satisfies - a -h,i,j n _ r lim co n- tal and lim n- nr G tal 2 ~ -1 X. 2 X. = 0 , t,1 n,1 for i = 1, 2, ..• , p; j - 1, 2, ••• , p , where o = ..n diag{(~ x2 .\ ~ \(=1 t~ Then, it can be shown that , ... , ~n r X 2)~} t=l t,p -33- and Now we obtain the asymptotic properties of the least squares estimators for = (Y t - 1 • the case when ~t Theorem 3.5: Yt-2' ...• Yt - p )· Assume that X saisfies the conditions in Theorem 3.3. ~t and where B ,. ~3 [9 o ~ Q ~ for j ~ 1 = a 2 9-1 + 6 Q-1 Q Q-1 1~1 ~2~1 i. 2 (Q2)·· = E[Y .Y .e 1] ~ 1J t -1 t -J t CD k+i-l = y e 2(0)k=O 1: wkwk +· .6 1 J-1 Then. -34- and -1 = E[v t Yt -1.'Y t -].J ~ -1 2 = k~O wk wk+i - 1 E[v t et_i_kJ . Proof: Note that, -1 n E Y n .Y a.s. . tal t-1. t-J and hence a.s. Consider, S n a n'X'e -0- ... n = t=l Eb e t t where b a t n. OY . i=l 1., t-1. t and n_ is an arbitrary vector with n'n ~ O. Then S is a martingale with, -0n n n' X' X e 2 _ ~O vn2 = 60 -0 n' x'x nO + 6 a.s. , E -0 1 -t -t t 1 t=l O and s 2 n = n 6 n' gl~O + n 6 n' 1 -0 0 -0 Note that, for j n -1 n E Y .Y ~ E[~~~te~_lJ ~O i, e 2 t=l t-1. t-j t-l ~ = k=O E -1 n 2 E wkw s n E e . k e .e 1 s=O t=l t-1.- t-S-] t~ -35- Therefore, and n -1 2 sn - Since E[e:J is finite, the Lindeberg condition is satisfied and s -1 n L S -N(O,I) . n Therefore, Using the arguments similar to those of Theorem 3.4, it follows that n~(~-~)~(2' 2 ~~l~l~~l) n~(~-~)~(2' 2~;~ ~l~;~) , and a and ~ are asymptotically independent. Now, to obtain the limiting distribution of A ~, consider = where gn and v2 n ~O are as defined in Theorem 3.4. = nO' X'G-1Xn - -n --0 a. s., and From Theorem 3.3, we know that Then, S n is a martingale with -36- n -1 n -1 -1 r v Y .Y . -- E[v t Yt -L.Y~.J t=l t t -L t -J ~-J a.s. Therefore, s -2 2 n Ll V n . and n -1 2 n' E[v-1X'X J sn -- -0 t -t-t ~O· Using Scott's martingale central limit theorem, we get s -1 S n L -N(O,l) n and Note also that ~ and ~ are asymptotically independent. 0 If X involves both fixed and lagged variables then one can obtain results -t si.milar to those of Fuller, Hasza and Goebel(l981). Also, if Yt process has a unit root, we can obtain the asymptotic distribution of the least squares estimator. Consider, for example, and {e } satisfies the conditions of Theorem 3.4. t of a l The least squares estimator is given by, a l = [t~2 Y;-l] -1 Then, n 2 -1 n = [n - 2 t=2 r Y J [n- l r Y e J t-l t=2 t-l t -37- If 6 = 0, Dickey and Fuller (1979) obtained the asymptotic distribution of 1 n(a 1 - 1). We now show that even if 6 1 ~ 0, n(a l - 1) has the same limiting distribution. We know that .tim n n-- -1 n 2 r e t=2 t a.s. Now consider, T n = n -~ Y = n-l r n-1 a. z* 1.,n i,n i=l and -2 n r n ,. n r y2t-l t=2 where z* = (Z*l ,n ' ..• ,Z*n- 1 ,n ) , = M e -n "'n mit(n) = (i,t) - th element of = 2(2n-1) -!.. 2 ~n 1 Cos[4n-2)- (2t-l)(2i-1)'lf] and l a i, n = ith element of n -~(l , ... , l)M-n -38- Using Scott's martingale central theorem, it follows that, for any fixed k, where !k is an k x k identity matrix. Now using the arguments similar to Hasza (1977) and Pantula (1982) it follows that n(~l - 1) has the same limiting distribution as that obtained by Dickey and Fuller (1979). Similarly, the results for pth order ARCH models may be obtained. 4. Summary: We have considered linear regression models with autoregressive conditionally heteroscedastic errors, introduced by Engel (1982). representation for the first order ARCH errors. We have obtained a series We hav.e used the representation to derive the ergodic properties of the errors. Similar representation can be obtained for the qth (q> 1) order ARCH errors but are not presented here. A special 2 case where the conditional error variance is of the form 80 + 8 a.e . 1 j=l J t-J -1 -1 [q+l-j] will be considered elsewhere. a. = q or a. = 2[q(q+1)] t J , where J We have considered the maximum likelihood estimation of ARCH regression models. The maximum likelihood estimators do not have explicit algebraic form and are computed using iterative methods. We have shown that the maximum likelihood estimators are strongly consistent and asymptotically normal. We have also shown that the least squares estimator and an estimated generalized least squares estimator are asymptotically normal. (Y t For a random walk model = alY - + et,a = 1) with ARCH errors, we have shown that the asymptotic l t l distribution of the least squares estimator of a l is the distribution obtained by Dickey and Fuller (1979) for the homoscedastic case. Acknowledgements I wish to express my thanks to Professor Wayne A. Fuller for helpful suggestions. -39- BIBLIOGRAPHY Chung, K. L. (1974). A Course in Probability Theory. Crowder, M. J. (1976). Observations, Academic Press, NY. Maximum Likelihood Estimation for Dependent Journal of the Royal Statistical Society, Series B, 45-53 • . Dickey, D. A. and W. A. Fuller (1979). Distribution of the Estimators for Autoregressive Time Series with a Unit Root, Journal of American Statistical Association, 427-531. Z~, Engle, R. F. (1982). Autoregressive Conditional Heteroscedasticity with Estimates of United Kingdom Inflation, Econometrica, Fuller, W. A. (1976). 2Q, 987-1007. Introduction to Statistical Time Series. Fuller, W. A., D. P. Hasza, and J. J. Goebel (1981). Wiley, NY. Estimation of the Parameters of Stochastic Difference Equations, The Annals of Statistics, 2, 531-543. Granger, C. W. J. and A. Anderson (1978). An Introduction to Bilinear TimeSeries Models. Vandenhoeck and Ruprecht, Gtlttingen. Hall, P. and C. C. Heyde (1980). Martingale Limit Theory and Its Application. Academic Press, NY. Hasza, D. P. (1977). Estimation in Nonstationary Time Series. Unpublished Ph.D. Thesis, Iowa State University, Ames, Iowa. Jones, R. H. (1965). An Experiment in Nonlinear Prediction, Applied Meterology, Pantula, S. G. (1982). Time Series. ~, Journal of 701-705. Properties of Estimator of the Parameters of Autoregressive Unpublished Ph.D. Thesis, Iowa State University, Ames, Iowa. Priestly, M. B. (1978). The Statistician, Nonlinear Models in Time Series Analysis, ~Z' 159-176. -40- Revesz, P. (1968). Scott, D. J. (1973). The Laws of Large Numbers. Academic Press, NY. Central Limit Theorems for Martingales and Processes with Stationary Increments Using a Skorokhod Representation Approach, Advances in Applied Probability, ~, 119-137.