Law of Large Number

Chebyshev’s inequality

probability of being an outlier has an upper-bound

$$\begin{array}{rl}{\sigma }^2& =\int \_{-\infty }^{+\infty }(x-\mu {)}^{2}f(x),dx\\ & \ge \int \_|x-\mu |\ge ϵ(x-\mu {)}^{2}f(x),dx\\ & \ge {ϵ}^{2}P(|x-\mu |\ge ϵ)\end{array}$$

if outliers are identified by $k\sigma$, the probability is bounded by $\frac{1}{k^2}$:

$$P(|x-\mu |\ge k\sigma )\le \frac{1}{k^2}$$

Weak Law of Large Number

Sample Mean

mean of $n$ i.i.d. random variable

$$\overline{X\_n}=\frac{1}{n}\sum \_{i=1}^{n}X\_i$$

has the properties:

• it’s expectation is: $\mu$
• It’s variance is: $\frac{{\sigma }^2}{n}$

by Chebyshev’s inequality,

$$P(|X-\mu |\ge ϵ)\le \frac{{\sigma }^2}{n{ϵ}^2}$$

as $n\to +\infty$, $X$ converges to $\mu$ in probability.

$$\lim \_n\to \infty P\left(|X-\mu |<ϵ\right)\le \lim \_n\to \infty \frac{{\sigma }^2}{n{ϵ}^2}=0$$

the probability of being an outlier would be very low.

Why it’s called “weak”: it converges in probability, which is weaker than almost sure converges, stated by SLLN

Strong Law of Large Number

$$P\left(\lim \_n\to \infty \bar{X}\_n=\mu \right)=1$$

the sample mean definitely converges to the expectation as $n$ growths.