Correlated Sampling

Consider the task of computing the integral

$$\Delta I= I_1 - I_2,$$ (348)

with

$$I_1 =\int dx g_1(x) f_1(x),$$ (349)

and

$$I_2 =\int dx g_2(x) f_2(x).$$ (350)

The procedure for correlated sampling can be described as follows:

Step (1). Sample random configurations $x_1, ..., x_N$ by using the sampling function $f_1(x)$ and evaluate the function $g_1$ for each of these configurations to obtain $g_1(x_1)$, $g_1(x_2)$, $g_1(x_3)$ ... $g_1(x_N)$. In addition, sample random configurations $y_1, ..., y_N$ by using the sampling function $f_2(y)$ and evaluate the function $g_2$ for each of these configurations to obtain $g_2(y_1)$, $g_2(y_2)$, $g_2(y_3)$ ... $g_2(y_N)$.
Step (2) Estimate $\Delta I$ according to

$$\Delta I = \frac{1}{N} \sum_{j=1}^{N} g_1(x_j) - g_2(y_j).$$ (351)

The variance of $\Delta I$ is

$$\sigma^2 = \frac{1}{N} \sum_{j=1}^{N} \Bigg( g_1(x_j) - g_2(y_j) - (I_1-I_2) \Bigg)^2,$$ (352)

or

$$\sigma^2 = \frac{1}{N} \sum_{j=1}^{N} \Bigg( g_1(x_j) - I_1 \Bigg......1}{N} \sum_{j=1}^{N} \Bigg( g_1(x_j) - I_1 \Bigg) \Bigg( g_2(y_j) - I_2 \Bigg),$$ (353)

where the first two terms on the r.h.s. of Eq. (353) are the variances $\sigma_1^2$ and $\sigma_2^2$ of the random variables $g_1$ and $g_2$, respectively, and the third term is the covariance cov(g1,g2) of the two random variables. Note that when $x_j$ and $y_j$ are statistically independent then the cov(g1,g2)=0 and

$$\sigma^2 = \sigma_1^2 + \sigma_2^2.$$ (354)

However, if the random variables are postively correlated then the cov$(g1,g2)> 0$ and the variance $\sigma^2$ is reduced.

The key to reduce the variance is thus to insure positive correlation between $g_1$ and $g_2$. This could be achieved by using the same sequence of random numbers for sampling both sets of random configurations $x_j$ and $y_j$.