Endogeneity and IV Estimation

Endogeneity happens when exogeneity is violated, i.e., $E[\epsilon |X]\ne 0$.

The following are some sources of endogeneity.

• Omitted variable bias
• Simultaneous/Reverse causality
• Measurement error
• Sample selection bias

Assumption

$[x_i,z_i,{\epsilon }_{𝑖}],𝑖=1,\dots ,𝑛$ are an i.i.d. sequence of random variables.

Explanatory variables

When exogeneity is violated, we assume

$$E[\epsilon |X]=\eta ,$$

($\eta$ is a function of $X$) and

$$E[x_i{\epsilon }_i]=\gamma \stackrel{\mathrm{辛}\mathrm{钦}\mathrm{大}\mathrm{数}\mathrm{定}\mathrm{理}}{⟹}\mathrm{plim}\frac{1}{n}{X}^{\prime }\epsilon =\gamma ,$$

then the estimator $b$ is biased,

$$E[b|X]=\beta +(X^{\prime }X{)}^{-1}{X}^{\prime }\eta \ne \beta ,$$

and is inconsistent,

$$\mathrm{plim}𝑏=\beta +\mathrm{plim}{\left(\frac{X^{\prime }X}{𝑛}\right)}^{-1}\mathrm{plim}\left(\frac{X^{\prime }\epsilon }{𝑛}\right)=\beta +{𝑄}_{XX}^{-1}\gamma \ne \beta .$$

Instrumental variables

The instrumental variable should satisfy:

1. Exogeneity. They are uncorrelated with the disturbance $\epsilon$.
2. Relevance. They are correlated with the independent variable $X$.

We can also describe it as:

1. $\mathrm{plim}\frac{1}{n}{Z}^{\prime }Z=Q_{ZZ}$, a finite, positive definite matrix (well-behaved data).
2. $\mathrm{plim}\frac{1}{n}{𝑍}^{\prime }X={𝑄}_{𝑍𝑋}$, a finite $L\times K$ matrix with rank $K$ (relevance).
3. $\mathrm{plim}\frac{1}{n}{𝑍}^{\prime }\epsilon =0$ (exogeneity).

IV Estimation

Situation when L=K

We partition $X$ into ${𝑥}_1$, a set of ${𝐾}_1$ exogenous variables, and ${𝑥}_2$, a set of ${𝐾}_2$ endogenous variables, then $𝑍=[{𝑥}_1,{𝑧}_2]$, where ${𝑧}_2$ are the instrumental variables for ${𝑥}_2$, and ${𝑥}_1$ are the instrumental variables for themselves.

$${𝑏}_{𝐼𝑉}=({𝑍}^{\prime }𝑋{)}^{-1}{𝑍}^{\prime }𝑦.$$

$b_{IV}$ is consistent.

• The asymptotic distribution is

  $$b_{IV}\stackrel{a}{\sim }N[\beta ,\frac{{\sigma }^2}{n}{Q}_{ZX}^{-1}{Q}_{ZZ}{Q}_{XZ}^{-1}].$$

• The asymptotic covariance matrix is estimated as

  $$\mathrm{Est}.\mathrm{Asy}.\mathrm{Var}[b_{IV}]={\hat{\sigma }}^2(Z^{\prime }X{)}^{-1}(Z^{\prime }Z)(X^{\prime }Z{)}^{-1},$$

  where

  $${\hat{\sigma }}^2=\frac{1}{n}\sum _{i=1}^n(y_i-x_i^{\prime }{b}_{IV}{)}^2.$$

In general, we have

$${𝑄}_{𝑋𝑦}={𝑄}_{𝑋𝑋}\beta +\gamma ,$$

while $\beta$ and $\gamma$ cannot be jointly identified.

• Assume $\gamma =0$, which is the standard OLS assumption.
• Find instrumental variables $Z$, use $Q_{𝑍𝑦}={𝑄}_{𝑍𝑋}\beta$ to estimate $\beta$.

Situation when L>K

Estimator:

$$\begin{array}{rl}{b}_{IV}& =({\hat{X}}^{\prime }X{)}^{-1}{\widehat{𝑋}}^{\prime }𝑦\\ & =[{𝑋}^{\prime }𝑍({𝑍}^{\prime }𝑍{)}^{-1}{𝑍}^{\prime }𝑋{]}^{-1}{𝑋}^{\prime }𝑍({𝑍}^{\prime }𝑍{)}^{-1}{𝑍}^{\prime }𝑦,\end{array}$$

where $\widehat{𝑋}=𝑍({𝑍}^{\prime }𝑍{)}^{-1}{𝑍}^{\prime }𝑋$.

$\widehat{𝑋}$ is the most efficient IV.

In practice, ${𝑏}_{𝐼𝑉}$ can be estimated in two steps: first estimate $\widehat{𝑋}$, and then ${𝑏}_{𝐼𝑉}=({\hat{X}}^{\prime }\hat{X}{)}^{-1}{\hat{X}}^{\prime }y$.

$$\begin{gathered} \boxed{{\displaystyle {\hat{X}}^{\prime }\hat{X}={𝑋}^{\prime }𝑍({𝑍}^{\prime }𝑍{)}^{-1}{𝑍}^{\prime }𝑍({𝑍}^{\prime }𝑍{)}^{-1}{𝑍}^{\prime }𝑋 \\ ={𝑋}^{\prime }𝑍({𝑍}^{\prime }𝑍{)}^{-1}{𝑍}^{\prime }X={\hat{X}}^{\prime }X.}} \end{gathered}$$

Relevant test

We want to test whether the regressors are correlated with the disturbances.

Hausman Test

The statistic is

$$H=\frac{{(b_{IV}-b_{LS})}^{\prime }{[{\left({\hat{X}}^{\prime }\hat{X}\right)}^{-1}-{\left(X^{\prime }X\right)}^{-1}]}^{-1}(b_{IV}-b_{LS})}{s^2}\sim {\chi }^2\left(K^{\ast }\right),$$

where $K^{\ast }$ is the amount of endogenous variables.

Hausman Statistic

The covariance between an efficient estimator ${𝑏}_E$ of a parameter vector $\beta$ and its difference from an inefficient estimator ${𝑏}_{𝐼}$ of the same parameter vector ${𝑏}_{𝐸}-{𝑏}_{𝐼}$ is zero,

• which means$$\mathrm{Cov}(b_E,b_E-b_I)=0.$$
• Thus$$\mathrm{Cov}({𝑏}_{𝐸},{𝑏}_{𝐼})=\mathrm{Var}({𝑏}_{𝐸}).$$

We have a pair of estimators ${\hat{\theta }}_E$ and ${\hat{\theta }}_I$, such that under ${𝐻}_0:{\hat{\theta }}_E$ and ${\hat{\theta }}_I$ are both consistent and ${\hat{\theta }}_E$ is efficient relative to ${\hat{\theta }}_I$, while under $H_1:{\hat{\theta }}_I$ remains consistent while ${\hat{\theta }}_E$ is inconsistent, then we can form a test of the hypothesis by the Hausman Statistic

$$H={({\hat{\theta }}_I-{\hat{\theta }}_E)}^{\prime }{\{\text{ Est. Asy. }\mathrm{Var}\left[{\hat{\theta }}_I\right]-\text{ Est. Asy. }\mathrm{Var}\left[{\hat{\theta }}_E\right]\}}^{-1}({\hat{\theta }}_I-{\hat{\theta }}_E)\stackrel{d}{\to }{\chi }^2(J).$$

Problems

Weak Instrument

The asymptotic covariance matrix of the IV estimator,

$$\text{ Est. Asy. }\mathrm{Var}\left[b_{IV}\right]={\hat{\sigma }}^2{\left(Z^{\prime }X\right)}^{-1}\left(Z^{\prime }Z\right){\left(X^{\prime }Z\right)}^{-1},$$

will be “large” when the instruments are weak.

Measurement Error

Single Regressor Model

A regression model with a single regressor and no constant term,

$${𝑦}^{\ast }=\beta {𝑥}^{\ast }+\epsilon ,$$

while ${𝑦}^{\ast }$ and ${𝑥}^{\ast }$ are not available, we can only observe $y$ and $x$,

$$𝑦={𝑦}^{\ast }+𝑣\text{ with }𝑣\sim 𝑁[0,{\sigma }_{𝑣}^2],$$$$𝑥={𝑥}^{\ast }+𝑢\text{ with }𝑢\sim 𝑁[0,{\sigma }_{𝑢}^2].$$

While the measurement error on ${𝑦}^{\ast }$ can be ignored, the measurement error on $x^{\ast }$ causes

$$\mathrm{plim}𝑏=\frac{\beta }{1+{\sigma }_{𝑢}^2/{𝑄}^{\ast }},$$

where ${𝑄}^{\ast }=\mathrm{plim}\frac{1}{𝑛}\sum _{𝑖}{𝑥}_{𝑖}^{\ast 2}$.

Multiple Regression Model

In a multiple regression model,

$$\mathrm{plim}b={[Q^{\ast }+{\mathrm{\Sigma }}_{uu}]}^{-1}{Q}^{\ast }\beta =\beta -{[Q^{\ast }+{\mathrm{\Sigma }}_{uu}]}^{-1}{\mathrm{\Sigma }}_{uu}\beta .$$

When only a single variable is measured with error (assume the first variable), we have

$$\mathrm{plim}{b}_1=\frac{{\beta }_1}{1+{\sigma }_u^2{q}^{\ast 11}},$$

and for $k\ne 1$,

$$\mathrm{plim}{b}_k={\beta }_k-{\beta }_1\frac{{\sigma }_u^2{q}^{\ast k1}}{1+{\sigma }_u^2{q}^{\ast 11}},$$

where $q^{\ast k1}$ is the $(k,1)$th element in $(Q^{\ast }{)}^{-1}$.