Algebraic formula for the variance

From a home for articles deleted from Wikipedia
Jump to: navigation, search
This article was considered for deletion at Wikipedia on November 21 2019. This is a backup of Wikipedia:Algebraic_formula_for_the_variance. All of its AfDs can be found at Wikipedia:Special:PrefixIndex/Wikipedia:Articles_for_deletion/Algebraic_formula_for_the_variance, the first at Wikipedia:Wikipedia:Articles_for_deletion/Algebraic_formula_for_the_variance. Purge

Original short description: "formulae for variance, some of which may in practice be quicker or more numerically stable to compute"

In probability theory and statistics, there are several algebraic formulae for the variance available for deriving the variance of a random variable. The usefulness of these depends on what is already known about the random variable; for example a random variable may be defined in terms of its probability density function or by construction from other random variables. The context here is that of deriving algebraic expressions for the theoretical variance of a random variable, in contrast to questions of estimating the variance of a population from sample data for which there are special considerations in implementing computational algorithms.

In terms of raw moments

If the raw moments E(X) and E(X 2) of a random variable X are known (where E(X) is the expected value of X), then Var(X) is given by

<math>\operatorname{Var}(X) = \operatorname{E}(X^2) - [\operatorname{E}(X)]^2 .</math>

The result is called the KönigHuygens formula in French-language literature[1] and known as Steiner translation theorem in Germany.[2]

There is a corresponding formula for use in estimation of the variance from sample data, that can be of use in hand calculations. This is a closely related identity that is structured to create an unbiased estimate of the population variance


\hat{\sigma}^2 = \frac{1}{N-1}\sum_{i=1}^N(x_i-\bar{x})^2 = \frac{N}{N-1}\left(\frac{1}{N}\left(\sum_{i=1}^N x_i^2\right) - \bar{x}^2\right) \equiv \frac{1}{N-1}\left(\left(\sum_{i=1}^N x_i^2\right) - N \left(\bar{x}\right)^2\right) . </math>

However, use of these formulas can be unwise in practice when using floating point arithmetic with limited precision: subtracting two values having a similar magnitude can lead to catastrophic cancellation,[3] and thus causing a loss of significance when <math>\operatorname{E}(X)^2 \gg \operatorname{Var}(X)</math>.[4] There several other numerically stable algorithms for calculating variance for use with floating point numbers.[4]


The computational formula for the population variance follows in a straightforward manner from the linearity of expected values and the definition of variance:


\begin{align} \operatorname{Var}(X)&=\operatorname{E}\left[(X - \operatorname{E}(X))^2\right]\\

                    &=\operatorname{E}\left[X^2 - 2X\operatorname{E}(X) + [\operatorname{E}(X)]^2\right]\\
                    &=\operatorname{E}(X^2) - \operatorname{E}[2X\operatorname{E}(X)] + [\operatorname{E}(X)]^2\\
                    &=\operatorname{E}(X^2) - 2\operatorname{E}(X)\operatorname{E}(X) + [\operatorname{E}(X)]^2\\
                    &=\operatorname{E}(X^2) - 2[\operatorname{E}(X)]^2 + [\operatorname{E}(X)]^2\\
                    &=\operatorname{E}(X^2) - [\operatorname{E}(X)]^2

\end{align} </math>

Generalization to covariance

This formula can be generalized for covariance, with two random variables Xi and Xj:

<math>\operatorname{Cov}(X_i, X_j) = \operatorname{E}(X_iX_j) -\operatorname{E}(X_i)\operatorname{E}(X_j)</math>

as well as for the n by n covariance matrix of a random vector of length n:

<math> \operatorname{Var}(\mathbf{X}) = \operatorname{E}(\mathbf{X X^\top}) - \operatorname{E}(\mathbf{X})\operatorname{E}(\mathbf{X})^\top</math>

and for the n by m cross-covariance matrix between two random vectors of length n and m:


\operatorname{Cov}(\textbf{X},\textbf{Y})= \operatorname{E}(\mathbf{X Y^\top}) - \operatorname{E}(\mathbf{X})\operatorname{E}(\mathbf{Y})^\top</math>

where expectations are taken element-wise and <math>\mathbf{X}=\{X_1,X_2,\ldots,X_n\}</math> and <math>\mathbf{Y}=\{Y_1,Y_2,\ldots,Y_m\}</math> are random vectors of respective lengths n and m.

Note that this formula suffers from the same loss of significance as the formula for variance if used for calculating estimates of the covariance, and alternative algorithms should be used instead.[4]

See also


  1. In French: formule de Koenig–Huygens. See e.g. Martiano, Jean-Jacques (2006), Maths: prépas commerciales, Studyrama, p. 148, ISBN 9782844728289 
  2. In German: Verschiebungssatz von Steiner. See e.g. Christoph, Gerd; Hackel, Horst (2013), Starthilfe Stochastik: Studium, Springer, p. 50, ISBN 9783322847997, .
  3. Donald E. Knuth (1998). The Art of Computer Programming, volume 2: Seminumerical Algorithms, 3rd edn., p. 232. Boston: Addison-Wesley.
  4. 4.0 4.1 4.2 Schubert, Erich; Gertz, Michael (2018-07-09). Numerically stable parallel computation of (co-)variance. ACM. pp. 10. Template:Citation error. ISBN 9781450365055.