Metropolis sampling example

Want to sample from a bivariate Gaussian with correlation. Metropolis sampling with a spherical Gaussian proposal:

Component x1 only:

Conservative proposals make no progress. Radical proposals get rejected. Want something in between.
Principle: Minimize the correlation coefficient of consecutive samples.







Suppose we want to estimate the mean of x1.
The naive estimate of Monte Carlo error doesn't work since samples are correlated.
Use an autoregressive model to correct it.

Variance of Monte Carlo estimate = var(f)/n * (1 + 2*sum_s rho(s)) (without any assumptions)
Variance of Monte Carlo estimate = var(f)/n * (1+rho(1))/(1-rho(1)) (under the autoregressive assumption: rho(s) = rho(1)^s)
rho(s) = cov(f(x_t), f(x_{t+s}))/var(f)




Variance of the Monte Carlo estimate





















Conditional density averaging






















Tom Minka
Last modified: Tue Jul 19 12:57:15 GMT 2005