Mean of Gaussians

Ali Rahimi (ali AT mit.edu) (last modified Jan 23, 2000)


1.0 What

Given a bunch of independent data points drawn from a gaussian, it's easy to estimate the maximum likelihood mean of the gaussian. It's just the average:
u_iid = sum(x_i, i)

However, if you happen to know that the data points are NOT IID (independent and identically distributed) and know their covariance, you can come up with a much better estimate of the mean:

u_ML = sum( invCov_ij x_i, ij) / sum(invCov_ij,ij)

where invCov is the inverse of Cov, and [Cov_ij] is a block matrix describing the covariance of data point x_i and x_j.

Here's a dramatic example you can try: draw 100 iid data points from a zero mean gaussian distribution. Assume that the correlation of data point x_i with point x_j is C_ij. Now add a constant (here, I use 3). Compute the mean assuming the data points are IID: 


>> mean(data)
ans =
    1.8457
Not very close to 3. Now use the above formula: 

>> sum(inv(C)*data) / sum(sum(inv(C)))
ans =
    3.0617
Much better.

2.0 Derivation