Bayesian Logistic Models with Pólya–Gamma Variables

Posted on Jun 10, 2026

Tags: Logistic Model

This note is for Polson, N. G., Scott, J. G., & Windle, J. (2013). Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables. Journal of the American Statistical Association, 108(504), 1339–1349.

Bayesian Inference for Logistic Models using Polya-Gamma Latent Variables

• Bayesian Inference for the logistic regression is hard, in contrast, Bayesian Inference for the probit model is much easier, owing to the simple latent-variable method

  the proposed method works for any binomial likelihood parameterized by log odds

A random variable $X$ has a Polya-Gamma distribution with parameters $b > 0$ and $c\in \IR$, denoted as $X\sim PG(b, c)$

the success of this strategy depends upon the existence of a simple, effective way to simulate Polya-Gamma random variables.

An accept/reject sampler based on the alternating-series method of Devroye (1986)

2 The Polya-Gamma Distribution

2.1 The Case PG(b, 0)

The Polya-Gamma family of distributions, denoted as $PG(b, c)$, is a subset of the class of infinite convolutions of gamma distributions.

PG(1, 0): also known as Polya distribution. It has Laplace distribution $\cosh^{-1}(\sqrt{t/2})$

using it as a starting point, one may define the random variable $w\sim PG(b, 0), b>0$ as the infinite convolution of gamma distributions that has Laplace transform

\[\bbE[\exp(-wt)] = \frac{1}{\cosh^b(\sqrt{t/2})}\]

by inverting the Laplace transform, one finds if $w\sim PG(b, 0)$, then it is equal in distribution to an infinite sum of gammas

2.2 The General PG(b, c) Class

the general PG(b, c) class arises through an exponential tilting of the PG(b, 0) density, much in the same way that a Gaussian likelihood combines with a Gamma prior for a precision.

2.3 Further Properties

• the density of a Polya-Gamma random variable can be expressed as an alternating-sign sum of inverse-Gaussian densities
• a further useful fact is that all finite moments of a Polya-Gamma random variable are available in a closed form. This allows the Polya-Gamma scheme to be used in EM algorithms
• the Polya-Gamma class is closed under convolution for random variables with the same tilting parameter. If $w_1\sim PG(b_1, z)$ and $w_2\sim PG(b_2, z)$ are independent, then $w_1+w_2\sim PG(b_1+b_2, z)$