WeiYa's Work Yard

A dog, who fell into the ocean of statistics, tries to write down his ideas and notes to save himself.

Bayesian Analysis of Mixture Distribution

Posted on June 27, 2017 0 Comments


Bayesian Models for Mixtures

Basic Formulation

$y_i, i=1,2,…,n$ have i.i.d. distribution

the objective is to infer the number $k$ of components.

an alternative representation:

given the values of group label $z_i$, the observations are draw from their respective individual subpopulations:

where $\theta_r=(\mu_{1:r},\lambda_{1:r},w_{1:r}), 2\le 2<\infty$.

Hierarchical Model and Priors

In a Bayesian framework, the unknown $k, w, \theta$ are regarded as drawn from appropriate prior distributions.

it is natural to impose further conditional independences, so

so we have the Bayesian hierarchical model

for full flexibility, add an extra layer to the hierarchy, and allow the priors for $k,w$ and $\theta$ to depend on hyperparameter $\lambda,\delta,\eta$ respectively.

Normal Mixtures


prior distribution are that the $\mu_j$ and $\sigma_j^{-2}$ are all drawn independently, with normal and gamma priors

so $\eta=(\xi,\kappa,\alpha,\beta)$

the prior on $w$ will always be taken as symetric Dirichlet,

a proper prior distribution for $k$ and a common choice is the Poisson distribution with hyperparameter $\lambda$

Reversible Jump Markov Chain Monte Carlo Algorithm for Mixtures


Weak Prior Information for Component Parameters

it seems restrictive to suppose that knowledge of the range of data implies much about the size of the $\sigma_j^2$, so introduce an additional hierarchical level by allowing $\beta$ to follow a gamma distribution with parameter $g$ and $h$. Generally take $\alpha >1>g$ to express the belief that $\sigma_j^2$ are similar, the scale parameter $h$ will be a small multiple of $1/R^2$.

prior distributions

same for each component $j=1,…,r$


Richardson, Sylvia, and Peter J. Green. “On Bayesian analysis of mixtures with an unknown number of components (with discussion).” Journal of the Royal Statistical Society: series B (statistical methodology) 59.4 (1997): 731-792.

Published in categories Bayesian Inference