MAST30020 Probability for Inference
Contents of the Course: an Outline
1
Probability spaces
Random experiments (1). Sample space. Product spaces. Examples. Indicator
functions (8). Expressing events using set operations. σ-algebras (10). Generated σ-algebras. Borel subsets. Probability (16). Probability space. Examples.
Elementary properties of probability (20). Continuity of probability (23). BorelCantelli lemma (26).
Bonferroni inequalities.1
2
Probabilities on R
Distribution functions (DFs) and their basic properties (27). Discrete probabilities.
Absolutely continuous distributions (35). Mixed and singular distributions.
Elementary conditional probabilities (given an event). Convolution formula for discrete
distributions.
3
Random variables
Definition of a random variable (RV) (38). Generated σ-algebras. Examples. Simple RVs. Random vectors (RVecs). Combinations of RVs. Distributions and DFs
of RVs and RVecs (45). Discrete RVecs. Some popular distributions. Transformations of RVs (48). Quantile transform. Independent RVs (50). Criteria of
independence. Independent events.
Order statistics. The distributions of X(k) and (X(k) , X(m) ) for 1 ≤ k < m ≤ n, for an
i.i.d. sample X1 , . . . , Xn .
4
Expectations
Motivation via relative frequencies (56). Expectations of simple RVs and their
properties. Approximation of non-negative RVs by increasing sequences of simple RVs (59). Consistency of the definition of expectation as a limit of expectations of approximating simple RVs. Key properties of expectation (65). Integrals
w.r.t. DFs (68). Expectation of a RV as integral of the distribution tail(s). (71)
Functions of RVs. Expectation of the product of independent RVs. Moments.
Jensen’s, Lyapunov’s and Chebyshev’s inequalities (76). Mixed moments, covariance and correlation. Cauchy-Bunyakovsky inequality (80). Covariance as a scalar
product (82). Geometric interpretation of correlation (84). Covariance matrices
(CovMs). Multivariate normal distributions (88).
1
In small font are listed some of the topics reviewed/covered in prac classes.
1
The mean value EX as argmina E|X − a|. Computing moments by integrating the
distribution tails with power factors. Best linear predictors.
5
Conditional expectations
Motivation (90). Conditional expectation (CE) given a simple RV (94). The
general definition (96). Conditional distributions (100). Properties of CEs (103).
Geometric interpretation of CE (as a projection; 109).
Conditional densities. Computing conditional densities for components of normal vectors. Conditional distribution of X1 given X(n) ; given (X(1) , X(n) ) (for an i.i.d. sample).
6
Some applications to Statistics
On relationship between Probability Theory and Mathematical Statistics (110).
Statistics (112). The definition of a sufficient statistic. Densities. Neyman-Fisher
factorisation criterion (117). Examples. Maximum likelihood estimators (123).
Classes of estimators with a given bias. Uniqueness of efficient estimators. RaoBlackwell theorem (127). Examples.
Sufficiency of the vector of order statistics.
7
Convergence of random variables
Modes of convergence (132). Examples. Some relationships among the modes (139).
Examples and counterexamples. Convergence under transformations (145). The
Weak Law of Large Numbers for the Bernoulli scheme (150). The Strong Law of
Large Numbers for the Bernoulli scheme. Possible extensions (155).
8
Characteristic functions
The definition of characteristic function (ChF; 157). Examples and basic properties. Independence and ChFs (162). Moments and ChFs. Inversion formulae (164).
Examples. Continuity theorems (171). Applications: the Weak Law of Large Numbers (175), the Central Limit Theorem (CLT; 177), Poisson limit theorem (181).
Multivariate ChFs (183). The multivariate CLT. The CLT for multinomial distributions (187). The χ2 statistic (192). The t-distribution (196).
Generating functions (both uni- and multivariate).
9
Further applications to Statistics
Empirical distribution functions (197). Glivenko-Cantelli theorem (200). Distribution-free statistics. MLEs revisited (205). Gibbs’ inequality (208) and the maximum of the log-likelihood function. Consistency and asymptotic normality of
MLEs (212).
2