And now it’s all this

I just said what I said and it was wrong
Or was taken wrong

Previous post

Variance of a sum

November 13, 2025 at 12:07 PM by Dr. Drang

Earlier this week, John D. Cook wrote a post about minimizing the variance of a sum of random variables. The sum he looked at was this:

$$Z=tX+(1-t)Y$$

where $X$ and $Y$ are independent random variables, and $t$ is a deterministic value. The proportion of $Z$ that comes from $X$ is $t$ and the proportion that comes from $Y$ is $1-t$. The goal is to choose $t$ to minimize the variance of $Z$. As Cook says, this is weighting the sum to minimize its variance.

The result he gets is

$$t=\frac{\mathrm{V}\mathrm{a}\mathrm{r}(Y)}{\mathrm{V}\mathrm{a}\mathrm{r}(X)+\mathrm{V}\mathrm{a}\mathrm{r}(Y)}$$

and one of the consequences of this is that if $X$ and $Y$ have equal variances, the $t$ that minimizes the variance of $Z$ is $t=1/2$.

You might think that if the variances are equal, it shouldn’t matter what proportions you use for the two random variables, but it does. That’s due in no small part to the independence of $X$ and $Y$, which is part of the problem’s setup.

A natural question to ask, then, is what happens if $X$ and $Y$ aren’t independent. That’s what we’ll look into here.

---

First, a little review. The variance of a random variable, $X$, is defined as

$$\mathrm{V}\mathrm{a}\mathrm{r}(X)={\int }_{-\infty }^{\infty }(x-{\mu }_X{)}^2{f}_X(x)dx$$

where ${\mu }_X$ is the mean value of $X$ and $f_X(x)$ is its probability density function (PDF). The most familiar PDF is the bell-shaped curve of the normal distribution.

The mean value is defined like this:

$${\mu }_X={\int }_{-\infty }^{\infty }x{f}_X(x)dx$$

People often like to work with the standard deviation ${\sigma }_X$ instead of the variance. The relationship is

$$\mathrm{V}\mathrm{a}\mathrm{r}(X)={\sigma }_X^2$$

Now let’s consider two random variables, $X$ and $Y$. They have a joint PDF, $f_{XY}(x,y)$. The covariance of the two is defined like this:

$$\mathrm{C}\mathrm{o}\mathrm{v}(X,Y)={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }(x-{\mu }_X)(y-{\mu }_Y)f_{XY}(x,y)dxdy$$

It’s common to express the covariance in terms of the standard deviations and the correlation coefficient, $\rho$:

$$\mathrm{C}\mathrm{o}\mathrm{v}(X,Y)=\rho {\sigma }_X{\sigma }_Y$$

If we were going to deal with more random variables, I’d explicitly include the variables as subscripts to $\rho$, but there’s no need to in the two-variable situation.

The correlation coefficient is a pure number and is always in this range:

$$-1\le \rho \le 1$$

A positive value of $\rho$ means that the two variables tend to be above or below their respective mean values at the same time. A negative value of $\rho$ means that when one variable is above its mean, the other tends to be below its mean, and vice versa.

If $X$ and $Y$ are independent, their joint PDF can be expressed as the product of two individual PDFs:

$$f_{XY}(x,y)=f_X(x)f_Y(y)$$

which means

$$\begin{array}{ccc}\mathrm{C}\mathrm{o}\mathrm{v}(X,Y)& =& {\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }(x-{\mu }_X)(y-{\mu }_Y)f_X(x)f_Y(y)dxdy\\ & =& {\int }_{-\infty }^{\infty }(x-{\mu }_X)f_X(x)dx{\int }_{-\infty }^{\infty }(y-{\mu }_Y)f_Y(y)dy\\ & =& 0\end{array}$$

because of the definition of the mean given above. Cook took advantage of this in his analysis to simplify his equations. We won’t be doing that.

---

Going back to our definition of $Z$,

$$Z=tX+(1-t)Y$$

the variance of $Z$ is

$${\sigma }_Z^2=t^2{\sigma }_X^2+2t(1-t)\rho {\sigma }_X{\sigma }_Y+(1-t{)}^2{\sigma }_Y^2$$

To get the value of $t$ that minimizes the variance, we take the derivative with respect to $t$ and set that equal to zero. This leads to

$$t=\frac{{\sigma }_Y^2-\rho {\sigma }_X{\sigma }_Y}{{\sigma }_X^2-2\rho {\sigma }_X{\sigma }_Y+{\sigma }_Y^2}$$

This reduces to Cook’s equation when $\rho =0$, which is what we’d expect.

At this value of $t$, the variance of the sum is

$${\sigma }_Z^2=\frac{(1-{\rho }^2){\sigma }_X^2{\sigma }_Y^2}{{\sigma }_X^2-2\rho {\sigma }_X{\sigma }_Y+{\sigma }_Y^2}$$

Considering now the situation where ${\sigma }_Y={\sigma }_X$, the value of $t$ that minimizes the variance is

$$t=\frac{{\sigma }_X^2-\rho {\sigma }_X^2}{2{\sigma }_X^2-2\rho {\sigma }_X^2}=\frac{1}{2}$$

which is the same result as before. In other words, when the variances of $X$ and $Y$ are equal, the variance of their sum is minimized by having equal amounts of both, regardless of their correlation. I don’t know about you, but I wasn’t expecting that.

Just because the minimizing value of $t$ doesn’t depend on the correlation coefficient, that doesn’t mean the variance itself doesn’t. The minimum variance of $Z$ when ${\sigma }_Y={\sigma }_X$ is

$${\sigma }_Z^2=\frac{1}{2}(1+\rho ){\sigma }_X^2$$

A pretty simple result and one that I did expect. When $X$ and $Y$ are positively correlated, their extremes tend to reinforce each other and the variance of $Z$ goes up. When $X$ and $Y$ are negatively correlated, their extremes tend to balance out, and $Z$ stays closer to its mean value.

Previous post

Site search

Meta

• drdrang at leancrew
• Blog archive
• RSS feed
• JSON feed
• Mastodon
• GitHub repositories

Recent posts

Credits

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.

© 2005–2025, Dr. Drang