Variance

Variance of Single RV

$$\mathrm{Var}[X]:=\mathbb{E}[(X-\mu {)}^2]$$

applying Linearity and Distributivity w.r.t addition:

$$\begin{array}{rl}\mathrm{Var}[X]& =\mathbb{E}[X^2-2\mu X+{\mu }^2]\\ & =\mathbb{E}[X^2]-2\mu \mathbb{E}[X]+{\mu }^2\\ & =\mathbb{E}[X^2]-{\mu }^2\end{array}$$

Transformation

$Y=CX$

$${\sigma }_{CX}^2=C^2{\sigma }_X^2$$

Proof:

$$\begin{array}{rl}{\sigma }_Y^2& =E[Y^2]-E^2[Y]\\ & =E[C^2{X}^2]-E[CX{]}^2\\ & =C^2(E[X^2]-E[X{]}^2)\\ & =C^2{\sigma }_X^2\end{array}$$

$Y=X+C$

Suppose $Y=X+C$, $C$ is a constant:

$${\sigma }_{X+C}^2={\sigma }_X^2$$

Proof 1:

Given (Proofs are in Linearity and Transformation):

$$\begin{array}{rl}{\mu }_Y& ={\mu }_X+C\\ f_Y(y)& =f_X(y-C)\end{array}$$

Thus:

$$\begin{array}{rl}{\sigma }_Y^2& =E_Y[(Y-{\mu }_Y{)}^2]\\ & ={\int }_{-\infty }^{\infty }(y-{\mu }_Y{)}^2{f}_Y(y)dy\\ & ={\int }_{-\infty }^{\infty }(y-{\mu }_Y{)}^2{f}_X(y-C)dy\\ & ={\int }_{-\infty }^{\infty }(s+C-({\mu }_X+C){)}^2{f}_X(s)ds\\ & =E_X\left[(X-{\mu }_X{)}^2\right]={\sigma }_X^2\end{array}$$

Proof2:

with LOTUS the proof is simpler:

$$\begin{array}{rl}{\sigma }_Y^2& =E[(Y-{\mu }_Y{)}^2]\\ & =E\left\{{\left[\left(X+C\right)-\left({\mu }_X+C\right)\right]}^2\right\}\\ & =E\left[{\left(X-{\mu }_X\right)}^2\right]={\sigma }_X^2\end{array}$$

Covariance

variance between two r.v.

$$\mathrm{Cov}(X,Y)=\mathbb{E}[(X-{\mu }_X)(Y-{\mu }_Y)]$$

also:

$$\begin{array}{rl}\mathrm{Cov}(X,Y)& =\mathbb{E}[XY-{\mu }_{X}Y-{\mu }_{Y}X+{\mu }_X{\mu }_Y]\\ & =\mathbb{E}[XY]-{\mu }_X{\mu }_y\end{array}$$

Covariance Matrix