Reference no: EM133291603
Econometric Analysis
Question 1. (OLS as MLE)
Consider the model
Y = XTβ + ε, (1)
where Y, ε ∈ Rn, X ∈ Rn×p, and β ∈ Rp. Here, n > 0 is the number of iid draws from the distributions of Y, X, and ε, and p < n is the number of covariates in this linear model. Also assume that ε ~ N(0, σ2I)|X for some σ2 > 0, where I is the n x n-identity matrix with ones on its main diagonal and zeros everywhere else. Note that this means that the conditional density fε|X(e) is Gaussian.
(a) Show that the log-likelihood function L (Y |X, β, σ2) ≡ L (β, σ2) takes the form
L (β,σ2) = C - 1/nlnσ2 - 1/2nσ2 (Y - Xβ)T (Y - Xβ)
for some finite constant C.
(b) Deduce that the optimal βˆ satisfies
1/2σ2n (XTY - XTX βˆ) = 0
(c) Deduce that βˆ = (XTX)-1XTY .
(d) Show that the optimal ˆσ2 satisfies
-1/2ˆσ2 + (Y-Xβ)T(Y-Xβ)/2ˆσ4n = 0.
(e) Deduce that
ˆσ2 = 1/n (Y - X βˆ)T (Y - X βˆ)
(f) Knowing that the variance of the OLS estimator ˆβOLS via linear regression is ˆεTˆε/n-p, show that ˆσ2 is a biased estimator with bias O(n-1).
(g) Deduce that the Maximum likelihood estimator βˆ does not have the smallest variance, i.e. is not efficient, among all estimators of the simple linear model (1).
(h) Deduce that βˆ is asymptotically efficient.
Question 2. The Cramér-Rao lower bound for general estimators of parametric models)
Let Z : Ω → Rd be a random variable with E[Z2] < +∞ which induces an absolutely continuous distribution with density function f (z|θ0), where θ0 ∈ R is the true (univariate) parameter indexing a specific class of density functions. We denote the support of the density f (z|θ0) by S and recall that f (z|θ0) > 0 everywhere on its support. We assume we observe an iid sample (Zi,. . . , Zn) of draws from the law of Z.
We write Z ~:= (Z1,. . . , Zn) to save on notation. In the following we consider some (not further specified) estimator θˆ ≡ T (Z) of the true θ0. We also assume the following regularity condition holds: there exist functions g(~z) ≡ g(z1, . . . ,zn) and ˜g(Z) such that
∫Rnd(˜g(Z→))2]dz→ < +∞, and |∂/∂θf(z|θ)| ≤ g (z) for all θ in a neighborhood U of θ0 as well as
We also assume that the variance
Var (∂/∂θ lnf(Z|θ) )θ=θ0 > 0,
where A(θ)|θ=θ0 means the expression A(θ) is evaluated at θ0.
(a) Argue that
E[T(Z→)] = ∫sT(Z→) Πni=1f(zi|θ0)dz→
(b) By considering the population maximum likelihood estimator
θ0 = argmax E [ln f(Z|θ)],
θ∈R
show that
E [∂θ lnf(Z|θ)]θ=θ0 = 0.
[Note that we consider Z here, not ~Z!]
(c) Based on these two steps deduce that
∂/∂θ E [T(Z)]θ=θ0 = Cov T(Z→), ∂/∂θ lnf(Z|θ)θ=θ0)
[Hint: Write out the expectations in the form of (a) and work "backwards" by noting that ∂/∂θ lnA(θ) = A(θ)-1∂/∂θ.A(θ).]
(d) Show that
(Cov ( T(Z→), ∂/∂θ lnf(Z→|θ)θ=θ0))2 ≤ nVar(T(Z→)).1/n Var (∂/∂θ ln f(Z→|θ)|θ=θ0)
(e) Deduce from this that
nVar(T(Z→)) ≥ (Var (∂/∂θ ln f(Z|θ)|θ=θ0)-1.(∂/∂θ E(T(Z→)θ=θ0)2
[Again, note that we use Z not Z→ in the first term!]
You have just derived the Cramér-Rao lower bound for the estimator T(Z→). Interpret this bound.
[Unrelated to this question: Note that
Var(∂/∂θ ln f(Z|θ))θ=θ0 = E [∂/∂θ lnf(Z|θ)|θ=θ0 ∂/∂θ lnf(Z|θ)|θ=θ0 ≡ I(θ)
is the Fisher information, which constitutes a Riemannian metric (i.e. an inner product on the tangent space at each point θ) on the manifold of the parametric class of density functions defined by the log likelihood. It is the fundamental concept in the area of information geometry.]
(f) Now assume that the estimator T(Z→) is unbiased, i.e. that E[T(Z→)] = θ0. Deduce that
nVar(T(Z→)) ≥ (Var (∂/∂θ lnf(Z|θ)θ=θ0))-1.
Interpret this inequality. What happens in the limit as n → ∞? How does this relate to the bound on the variance for finite n?