84 Section 8.5.Breakdown of assumptions Non-Existence of the MLE Multiple Solutions to Maximization Problem Multiple Solutions to Score Equations Number of Parameters Increase with the Sample Size Support of p(z;0)depends on 0 ●Non-I.I.D.Data
84 Section 8.5. Breakdown of assumptions • Non-Existence of the MLE • Multiple Solutions to Maximization Problem • Multiple Solutions to Score Equations • Number of Parameters Increase with the Sample Size • Support of p(x; θ) depends on θ • Non-I.I.D. Data
85 Non-Existence of the MLE The non-existence of the MLE may occur for all values of m or for only some of them.In general,this is due either to the fact that the parameter space is not compact or that the log-likelihood is discontinuous in 0. Example 8.1:Suppose that X~Bernoulli(1/(1+exp(0)),where e=R.If we observe z =1,then L(;1)=1/(1+exp(0)).The likelihood function is a decreasing function of 0 and the maximum is not attained on If were closed,i.e.,=R,the MLE would be -oo. Example 8.2:Suppose that X~Normal(u,o2).So,0=(u,o2) and日=R×R+.Now,l(0;x)ox-logo-a(z-)2.Take u=x.Then as o→0，l(0;x)→+o.So,the MLE does not exist
85 Non-Existence of the MLE The non-existence of the MLE may occur for all values of xn or for only some of them. In general, this is due either to the fact that the parameter space is not compact or that the log-likelihood is discontinuous in θ. Example 8.1: Suppose that X ∼ Bernoulli(1/(1 + exp(θ)), where Θ = R. If we observe x = 1, then L(θ; 1) = 1/(1 + exp(θ)). The likelihood function is a decreasing function of θ and the maximum is not attained on Θ. If Θ were closed, i.e., Θ = R ¯ , the MLE would be −∞. Example 8.2: Suppose that X ∼ Normal(µ, σ2). So, θ = (µ, σ2) and Θ = R × R+. Now, l(θ; x) ∝ − log σ − 12σ2 (x − µ)2. Take µ = x. Then as σ → 0, l(θ; x) → +∞. So, the MLE does not exist
86 Multiple Solutions One reason for multiple solutions to the maximization problem is non-identification of the parameter 0. Example 8.3:Suppose that Y~Normal(X0,I),where X is an n×k matrix with rank smaller than k and 0∈曰cRk.The density function is pv:0)-(2z)-a/2exp(-j(u-x0Y(v-X0) Since X is not full rank,there exists an infinite number of solutions to xo=0.That means that there exists an infinite number of 0's that generate the same density function.So,0 is not identified. Furthermore,note that the likelihood is maximized at all values of 0 satisfying X'X=X'y
86 Multiple Solutions One reason for multiple solutions to the maximization problem is non-identification of the parameter θ. Example 8.3: Suppose that Y ∼ Normal(Xθ, I), where X is an n × k matrix with rank smaller than k and θ ∈ Θ ⊂ Rk. The density function is p(y; θ) = (2π)−n/2 exp(−12(y − Xθ)(y − Xθ)) Since X is not full rank, there exists an infinite number of solutions to Xθ = 0. That means that there exists an infinite number of θ’s that generate the same density function. So, θ is not identified. Furthermore, note that the likelihood is maximized at all values of θ satisfying XXθ = Xy
87 Multiple Roots to the Score Equations Even though the score equations may have multiple roots for fixed n,we can still use our theorems to show consistency and asymptotic normality.This will work provided that as n gets large there is a unique maximum with large probability. Example 8.4:Suppose that Xn=(X1,...,Xn),where the Xi's are i.i.d.Cauchy(0,1).We assume that 0o lies in the interior of a compact setΘcR.So, 1 p(x;0)= π(1+(x-0)2) So,the log-likelihood for the full sample is l(0:x)=-nlogπ-∑log(1+(-0)2) i=1 Note that as0→±o,l(0;c)→-o
87 Multiple Roots to the Score Equations Even though the score equations may have multiple roots for fixed n, we can still use our theorems to show consistency and asymptotic normality. This will work provided that as n gets large there is a unique maximum with large probability. Example 8.4: Suppose that Xn = (X1,...,Xn), where the Xi’s are i.i.d. Cauchy(θ, 1). We assume that θ0 lies in the interior of a compact set Θ ⊂ R. So, p(x; θ) = 1 π(1 + (x − θ)2) So, the log-likelihood for the full sample is l(θ; x) = −n log π − n i=1 log(1 + (xi − θ)2) Note that as θ → ±∞, l(θ; x) → −∞
88 The score for 0 is given by 立部 2(xc-0) de =1 As the picture below demonstrates,there can be multiple roots to the score equations
88 The score for θ is given by dl(θ; x) dθ = n i=1 2(xi − θ) 1+(xi − θ)2 As the picture below demonstrates, there can be multiple roots to the score equations