Estimation of Location and Scale Parameters of Continuous Density
Posted on (Update: )
This note is for Pitman, E. J. G. (1939). The Estimation of the Location and Scale Parameters of a Continuous Population of any Given Form. Biometrika, 30(3/4), 391–421. and Kagan, AM & Rukhin, AL. (1967). On the estimation of a scale parameter. Theory of Probability \& Its Applications, 12, 672–678.
Pitman (1939)
The paper develops a general method of solving problems of estimation in which the unknown parameters are “location” and “scale” parameters. Suppose that the probability function of $X$ is
\[\frac 1cf\left(\frac{x-a}{c}\right)\]and that the function $f(x)$ is known but that one or both the parameters $a, c$, which determine respectively the location and the scale of the distribution of $X$, is unknown.
Denote by $H$ a function of the $\xi$ such that
\[E(H) = \int_W FHd\xi_1\cdots\xi_n\]exists. Let $W_+$ be the region where $F$ is not zero.
The estimation of $a$
skip…
The estimation of $c$
Here the probability function of $X$ is
\[\frac 1c f(x/c), c > 0\]and the probability of the sample $\xi_1,\ldots,\xi_n$ is
\[F = c^{-n} f(\xi_1/c)\cdots f(\xi_n/c)\]If $X$ takes only positive values, consider $\log X$ and $\gamma = \log c$, then the pdf for $\log X$ is
\[e^{x-\gamma} f(e^{x-\gamma})\,,\]which reduces to the estimation of location parameter in the previous part.
A estimator $C(x_1,\ldots, x_n)$ must satisfy
- $C(x_1,\ldots, x_n) \ge 0$
- $C(\lambda x_1,\ldots, \lambda x_n) = \lambda C(x_1,\ldots,x_n), \lambda \ge 0$
Any function of this type will be called a $c$ estimator. Its logarithm, $G$, which will be a $\gamma$ estimator, will satisfy
\[G(\lambda x_1,\ldots, \lambda x_n) = G(x_1,\ldots, x_n) + \log \lambda, \lambda \ge 0\]and any function of this type will be called a $\gamma$ estimator.
- $R$: a half line or ray with one end at the origin
- any point which lies on some $R$ is called observable
define the mean value of $H$ on $R$ by
\[E_R[H] = \frac{\int_0^\infty FHr^{n-1}dr}{\int_0^\infty Fr^{n-1}dr}\]where $r = \sqrt{\sum \xi_r^2}$.
If $E_R[H]$ has the same value $h$ on every $R$, then $E[H] = h$.
For a set of intervals $I’$ determined on $R$, define
\[P(I'\mid R) = \frac{\int_I Fr^{n-1}dr}{\int_0^\infty Fr^{n-1}dr}\]Let $w’$ be the region generated by $I’$, then $P(w’) = \alpha$ if $P(I’\mid R)=\alpha$.
If $(x_1,\ldots,x_n)$ is a fixed point on $R$, the co-ordinates $(\xi_1,\ldots,\xi_n)$ of any point on $R$ may be expressed in the form
\[e^{-\gamma}\xi_r = e^{-t}x_r\]The fiducial function for the estimation of $\gamma$ is
\[g_1(\gamma) = ke^{-n\gamma} f(x_1e^{-\gamma})\cdot f(x_n e^{-\gamma})\]The fiducial function for the estimation of $c$ is
\[g_2(c) = kc^{-n-1}f(x_1/c)\cdots f(x_n/c), c\ge 0\]The median of the fiducial distribution $G_C$ is the closest estimator of $\gamma$, and the estimator with the smallest mean absolute error.
\[G_M = E_g(\gamma) = \int_{-\infty}^\infty \gamma g_1(\gamma)d\gamma\]- $G_L$: the maximum likelihood estimator, is defined as the value of $\gamma$ which makes $g_1(\gamma)$ a maximum
- $G_{(r)}$: the estimator with the smallest mean $r$-th power absolute error.
The median $C_c$ is the closest estimator of $c$. Its median value is $c$, and its logarithm is $G_C$; but it is not in general the $c$ estimator with the smallest mean absolute error.
$C_{(2)}$, the $c$-estimator with the smallest mean square error, is defined by
\[C_{(2)} = \frac{E_g(1/c)}{E_g(1/c^2)}\,.\]Kagan & Rukhin (1967)
This part is based on Kagan, AM & Rukhin, AL. (1967). On the estimation of a scale parameter. Theory of Probability \& Its Applications, 12, 672–678.
consider a family of distribution function $F(x/\sigma)$ on the half-line $(0, \infty)$ indexed by a scale parameter $\sigma\in (0, \infty)$.
Assume that
\[\alpha_2 = \int_0^\infty x^2 dF(x) < \infty\]and let $\alpha_1=\int_0^\infty xdF(x)$, then the statistic
\[\alpha_1^{-1} \bar x\]will be an unbiased estimate of the parameter $\sigma$
Consider the class of estimates of the form
\[\sigma_n^*(x_1,\ldots,x_n) = c_n(\bar x)\]We have
\[E(\hat\sigma_n^\star - \sigma)^2= \sigma^2E(c_n\bar x - 1)^2\]and the minimum is attained when
\[c_n = \frac{\alpha_1}{\alpha_1^2 + (\alpha_2-\alpha_1^2)/n}\]Since $\alpha_2\ge \alpha_1^2$, and since equality is attained only for an improper d.f., it follows that in the class of all (biased and unbiased) estimates of the scale parameter $\sigma$, the unbiased estimate $\alpha_1^{-1}\bar x$ is always inadmissible (except in the trivial case of an improper $F(x)$)
Two questions:
- when is the estimate $\alpha_1^{-1}\bar x$ admissible in the class of unbiased estimates of the scale parameter $\sigma$
- when is the estimate $c_n^0\bar x$ admissible in the class of all estimates of the parameter $\sigma$; in other words, when is $c_n^0\bar x$ absoultely admissable?
Let $R$ be the class of proper estimates (scale invariant), then if $\tilde \sigma_n\in R$,
\[\tilde \sigma_n = \bar x \psi\left(\frac{x_2}{x_1},\ldots, \frac{x_n}{x_1}\right)\]Then the estimate
\[s_n = \tilde \sigma_n\frac{E(\tilde\sigma_n\mid y)}{E(\tilde \sigma_n^2\mid y)}\]is optimal in the class of proper estimates.
If $F(x) = \int_0^x f(u)du$, then
\[s_n(x_1,\ldots,x_n) = \frac{\int_0^\infty u^nf(ux_1)\cdots f(ux_n)du}{\int_0^\infty u^{n+1}f(ux_1)\cdots f(ux_n)du}\]The paper showed that $\alpha_1^{-1}\bar x$ will be admissible in the class of unbiased estimates of the parameter $\sigma$, and $c_n^0\bar x$ will be absolutely admissible only for the trivial case of an improper $F(x)$ and the gamma distribution.