Fisher's Linear Discriminant

class mean:

$$\mu \_j=\frac{1}{n\_j}\sum \_x\_i\in C\_jx\_i$$

class scatter:

$$S\_j=\sum \_x\_i\in C\_j(x\_i-\mu \_j)(x\_i-\mu \_j{)}^T$$

why not inner product?

goal: the find the optimal project w^{_} that maximize the between class distance and within class distance:

\begin{align} w^{_} & = \argmax\_{w} \frac{(m\_{1}- m\_{2})^{2}}{S\_{1}+S\_{2}} \\ m\_{j} & = w^{T}\mu\_{j} \\ s\_{j} & =w^{T}S\_{j}w \end{align}

within class scatter:

$$\begin{array}{rl}{w}^{\text{*}}& =\mathrm{argmax}\_w\frac{(m\_1-m\_2{)}^2}{S\_1+S\_2}\\ & =\mathrm{argmax}\_w\frac{(w^T\mu \_1-w^T\mu \_2{)}^2}{w^T(S\_1+S\_2)w}\\ & =\mathrm{argmax}\_w\frac{w^T(\mu \_1-\mu \_2)(\mu \_1-\mu \_2{)}^{T}w}{w^T(S\_1+S\_2)w}\\ & =\mathrm{argmax}\_w\frac{w^{T}S\_Bw}{w^{T}S\_ww}\end{array}$$