WeiYa's Work Yard

A dog, who fell into the ocean of statistics, tries to write down his ideas and notes to save himself.

Statistical Inference with Unnormalized Models

Posted on (Update: )
Tags: KL-divergence, Stein's Operator

This post is based on the talk given by T. Kanamori at the 11th ICSA International Conference on Dec. 22nd, 2019.

Problem Setting

Estimation Models

Other divergences: density-power divergences, pseudo spherical divergence, etc. => robust estimators

Different from the intractable models talked in the MCMC, which only aims to get a sample from the untractable model, but here we need to get a (MLE) estimate of the parameter.

Proposed Approach

where the variational representation of KL divergences is that

where the Stein operator is defined by

Theoretical Properties

more details can be found in the M-estimator.

the calculation procedure is as follows:

Numerical Experiments


Published in categories ICSA-2019