WeiYa's Work Yard

A dog, who fell into the ocean of statistics, tries to write down his ideas and notes to save himself.

Estimation of Location and Scale Parameters of Continuous Density

Posted on (Update: )
Tags: Equivariance, Location-Scale Family, Admissible

This note is for Pitman, E. J. G. (1939). The Estimation of the Location and Scale Parameters of a Continuous Population of any Given Form. Biometrika, 30(3/4), 391–421. and Kagan, AM & Rukhin, AL. (1967). On the estimation of a scale parameter. Theory of Probability \& Its Applications, 12, 672–678.

Pitman (1939)

The paper develops a general method of solving problems of estimation in which the unknown parameters are “location” and “scale” parameters. Suppose that the probability function of $X$ is

\[\frac 1cf\left(\frac{x-a}{c}\right)\]

and that the function $f(x)$ is known but that one or both the parameters $a, c$, which determine respectively the location and the scale of the distribution of $X$, is unknown.

Denote by $H$ a function of the $\xi$ such that

\[E(H) = \int_W FHd\xi_1\cdots\xi_n\]

exists. Let $W_+$ be the region where $F$ is not zero.

The estimation of $a$

skip…

The estimation of $c$

Here the probability function of $X$ is

\[\frac 1c f(x/c), c > 0\]

and the probability of the sample $\xi_1,\ldots,\xi_n$ is

\[F = c^{-n} f(\xi_1/c)\cdots f(\xi_n/c)\]

If $X$ takes only positive values, consider $\log X$ and $\gamma = \log c$, then the pdf for $\log X$ is

\[e^{x-\gamma} f(e^{x-\gamma})\,,\]

which reduces to the estimation of location parameter in the previous part.

A estimator $C(x_1,\ldots, x_n)$ must satisfy

  • $C(x_1,\ldots, x_n) \ge 0$
  • $C(\lambda x_1,\ldots, \lambda x_n) = \lambda C(x_1,\ldots,x_n), \lambda \ge 0$

Any function of this type will be called a $c$ estimator. Its logarithm, $G$, which will be a $\gamma$ estimator, will satisfy

\[G(\lambda x_1,\ldots, \lambda x_n) = G(x_1,\ldots, x_n) + \log \lambda, \lambda \ge 0\]

and any function of this type will be called a $\gamma$ estimator.

  • $R$: a half line or ray with one end at the origin
    • any point which lies on some $R$ is called observable

define the mean value of $H$ on $R$ by

\[E_R[H] = \frac{\int_0^\infty FHr^{n-1}dr}{\int_0^\infty Fr^{n-1}dr}\]

where $r = \sqrt{\sum \xi_r^2}$.

If $E_R[H]$ has the same value $h$ on every $R$, then $E[H] = h$.

For a set of intervals $I’$ determined on $R$, define

\[P(I'\mid R) = \frac{\int_I Fr^{n-1}dr}{\int_0^\infty Fr^{n-1}dr}\]

Let $w’$ be the region generated by $I’$, then $P(w’) = \alpha$ if $P(I’\mid R)=\alpha$.

If $(x_1,\ldots,x_n)$ is a fixed point on $R$, the co-ordinates $(\xi_1,\ldots,\xi_n)$ of any point on $R$ may be expressed in the form

\[e^{-\gamma}\xi_r = e^{-t}x_r\]

The fiducial function for the estimation of $\gamma$ is

\[g_1(\gamma) = ke^{-n\gamma} f(x_1e^{-\gamma})\cdot f(x_n e^{-\gamma})\]

The fiducial function for the estimation of $c$ is

\[g_2(c) = kc^{-n-1}f(x_1/c)\cdots f(x_n/c), c\ge 0\]

The median of the fiducial distribution $G_C$ is the closest estimator of $\gamma$, and the estimator with the smallest mean absolute error.

\[G_M = E_g(\gamma) = \int_{-\infty}^\infty \gamma g_1(\gamma)d\gamma\]
  • $G_L$: the maximum likelihood estimator, is defined as the value of $\gamma$ which makes $g_1(\gamma)$ a maximum
  • $G_{(r)}$: the estimator with the smallest mean $r$-th power absolute error.

The median $C_c$ is the closest estimator of $c$. Its median value is $c$, and its logarithm is $G_C$; but it is not in general the $c$ estimator with the smallest mean absolute error.

$C_{(2)}$, the $c$-estimator with the smallest mean square error, is defined by

\[C_{(2)} = \frac{E_g(1/c)}{E_g(1/c^2)}\,.\]

Kagan & Rukhin (1967)

This part is based on Kagan, AM & Rukhin, AL. (1967). On the estimation of a scale parameter. Theory of Probability \& Its Applications, 12, 672–678.

consider a family of distribution function $F(x/\sigma)$ on the half-line $(0, \infty)$ indexed by a scale parameter $\sigma\in (0, \infty)$.

Assume that

\[\alpha_2 = \int_0^\infty x^2 dF(x) < \infty\]

and let $\alpha_1=\int_0^\infty xdF(x)$, then the statistic

\[\alpha_1^{-1} \bar x\]

will be an unbiased estimate of the parameter $\sigma$

Consider the class of estimates of the form

\[\sigma_n^*(x_1,\ldots,x_n) = c_n(\bar x)\]

We have

\[E(\hat\sigma_n^\star - \sigma)^2= \sigma^2E(c_n\bar x - 1)^2\]

and the minimum is attained when

\[c_n = \frac{\alpha_1}{\alpha_1^2 + (\alpha_2-\alpha_1^2)/n}\]

Since $\alpha_2\ge \alpha_1^2$, and since equality is attained only for an improper d.f., it follows that in the class of all (biased and unbiased) estimates of the scale parameter $\sigma$, the unbiased estimate $\alpha_1^{-1}\bar x$ is always inadmissible (except in the trivial case of an improper $F(x)$)

Two questions:

  • when is the estimate $\alpha_1^{-1}\bar x$ admissible in the class of unbiased estimates of the scale parameter $\sigma$
  • when is the estimate $c_n^0\bar x$ admissible in the class of all estimates of the parameter $\sigma$; in other words, when is $c_n^0\bar x$ absoultely admissable?

Let $R$ be the class of proper estimates (scale invariant), then if $\tilde \sigma_n\in R$,

\[\tilde \sigma_n = \bar x \psi\left(\frac{x_2}{x_1},\ldots, \frac{x_n}{x_1}\right)\]

Then the estimate

\[s_n = \tilde \sigma_n\frac{E(\tilde\sigma_n\mid y)}{E(\tilde \sigma_n^2\mid y)}\]

is optimal in the class of proper estimates.

If $F(x) = \int_0^x f(u)du$, then

\[s_n(x_1,\ldots,x_n) = \frac{\int_0^\infty u^nf(ux_1)\cdots f(ux_n)du}{\int_0^\infty u^{n+1}f(ux_1)\cdots f(ux_n)du}\]

The paper showed that $\alpha_1^{-1}\bar x$ will be admissible in the class of unbiased estimates of the parameter $\sigma$, and $c_n^0\bar x$ will be absolutely admissible only for the trivial case of an improper $F(x)$ and the gamma distribution.


Published in categories Note