Large-Sample Distribution Theory

一般而言，计量经济学寻找的是想要了解的未知参数的一致估计量，即当样本量无穷大时，值能够无限趋近这一参数的估计量。

大数定律指出，一个期望 $\mu$ 和方差 ${\sigma }^2$ 均有限的随机样本$x$，其平均值 $\overline{x}$ 就是其期望 $\mu$ 的一致估计量。

另一方面，计量经济学需要判断参数估计的显著性，因此需要知晓估计量的标准差，从而需要考虑估计量的分布。

中心极限定理说明，独立同分布的样本可构造出一个依分布收敛的统计量。

依分布收敛和渐进分布表达的意思差不多。依分布收敛是理论，表达的是极限下的性质，渐进分布是应用，是有限样本下的实际使用。

Convergence in Probability

Definition (Convergence in Probability). The random variable $x_n$ converges in probability to a constant $c$ if

$$\underset{n\to \mathrm{\infty }}{lim}\mathrm{Prob}(|x_n-c|>\epsilon )=0\text{ for}\text{ any}\text{ positive }\epsilon .$$

• If $x_n$ converges in probability to $c$, then we write$$\mathrm{plim}{x}_n=c.$$

Theorem (Convergence in Quadratic Mean). If $x_n$ has mean ${\mu }_n$ and variance ${\sigma }_n^2$ such that the ordinary limits of ${\mu }_n$ and ${\sigma }_n^2$ are $c$ and $0$, respectively, $x_n$ converges in mean square to $c$, and

$$\mathrm{plim}{x}_n=c.$$

Definition (Consistent Estimator). An estimator ${\hat{\theta }}_n$ of a parameter $\theta$ is a consistent estimator of $\theta$ if and only if

$$\mathrm{plim}{\hat{\theta }}_n=\theta .$$

Theorem (Consistency of the Sample Mean). The mean of a random sample from any population with finite mean $\mu$ and finite variance ${\sigma }^2$ is a consistent estimator of $\mu$.

Laws of Large Numbers

Theorem (Khinchine’s Weak Law of Large Numbers). If $x_i,i=1,\cdots ,n$ is a random (i.i.d.) sample from a distribution with finite mean $E[x_i]=\mu$,

$$\mathrm{plim}{\overline{x}}_n=\mu$$

DANGER

$$\begin{array}{rl}E(x_i)=c& \Rightarrow \mathrm{plim}\frac{1}{n}\sum x_i=c\\ \mathrm{plim}\frac{1}{n}\sum x_i=c& \nRightarrow E(x_i)=c\end{array}$$

Theorem (Chebychev’s Weak Law of Large Numbers). If $x_i,i=1,\cdots ,n$ is a sample of observations such that $E[x_i]={\mu }_i<\mathrm{\infty }$ and $Var[x_i]={\sigma }_i^2<\mathrm{\infty }$ such that $\frac{{\overline{\sigma }}_n^2}{n}=(\frac{1}{n^2})\sum _i{\sigma }_i^2\to 0$ as $n\to \mathrm{\infty }$, then $\mathrm{plim}({\overline{x}}_n-{\overline{\mu }}_n)=0$.

Rules for Probability Limits

1. If $x_n$ and $y_n$ are random variables with $\mathrm{plim}{x}_n=c$ and $\mathrm{plim}{y}_n=d$, then

   $$\mathrm{plim}(x_n+y_n)=c+d\text{, (sum rule) }$$$$\mathrm{plim}{x}_n{y}_n=cd\text{, (product rule)}$$$$\mathrm{plim}{x}_n/y_n=c/d\text{ if }d\ne 0.\text{ (ratio rule)}$$

2. If $W_n$ is a matrix whose elements are random variables and if $\mathrm{plim}{W}_n=\mathrm{\Omega }$, then

   $$\mathrm{plim}{W}_n^{\text{-1}}={\mathrm{\Omega }}^{\text{-1}}.\text{ (matrix inverse rule)}$$

3. If $X_n$ and $Y_n$ are random matrices with $\mathrm{plim}{X}_n=A$ and $\mathrm{plim}{Y}_n=B$, then

   $$\mathrm{plim}{X}_n{Y}_n=AB.\text{ (matrix product rule)}$$

Convergence in Distribution

Definition (Convergence in Distribution). $x_n$ converges in distribution to a random variable $x$ with CDF $F(x)$ if $\underset{n\to \mathrm{\infty }}{lim}|F_n(x_n)-F(x)|=0$ at all continuity points of $F(x)$.

Definition (Limiting Distribution). If $x_n$ converges in distribution to $x$, where $F_n(x_n)$ is the CDF of $x_n$, then $F(x)$ is the limiting distribution of $x_n$. This is written as

$$x_n\stackrel{d}{\to }x$$

Rules for Limiting Distributions

$$x_n+y_n\stackrel{d}{\to }c+x,\text{ and }{x}_n/y_n\stackrel{d}{\to }x/c,\text{ if }c\ne 0.$$

If $x_n\stackrel{d}{\to }x$ and $g(x_n)$ is a continuous function, then $𝑔(x_n)\stackrel{d}{\to }𝑔(x)$.

If $y_n$ has a limiting distribution and $\mathrm{plim}(x_n-y_n)=0$, then $x_n$ has the same limiting distribution as $y_n$.

Central Limit Theorems

Theorem (Lindeberg-Levy Central Limit Theorem (Univariate)). If $x_1,\cdots ,x_n$ are a random sample from a probability distribution with finite mean \mu and finite variance ${\sigma }^2$ and ${\overline{x}}_n=(\frac{1}{n})\sum _{i=1}^n{x}_i$, then

$$\sqrt{n}({\overline{x}}_n-\mu )\stackrel{d}{\to }N[0,{\sigma }^2].$$

Asymptotic Distribution

Definition (Asymptotic Distribution). An asymptotic distribution is a distribution that is used to approximate the true finite sample distribution of a random variable.

If $\sqrt{n}[{\overline{x}}_n-\mu ]/\sigma \stackrel{d}{\to }N[0,1]$, then approximately, or asymptotically, ${\overline{x}}_n\sim N[\mu ,\frac{{\sigma }^2}{n}]$, which we write as

$$\overline{x}\stackrel{a}{\sim }N[\mu ,\frac{{\sigma }^2}{n}]$$