linalg review

$$\begin{array}{rl}{x}_1+x_2+x_3& =5\\ x_1-2x_2-3x_3& =-1\\ 2x_1+x_2-x_3& =3\end{array}$$

Equivalent matrix representation of $A\times x=b$

$$\begin{array}{rl}A& =\left(\begin{array}{ccc}1& 1& 1\\ 1& -2& -3\\ 2& 1& -1\end{array}\right)\\ X& =\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)\\ b& =\left(\begin{array}{c}5\\ -1\\ 3\end{array}\right)\end{array}$$

with $A\in R^{m\times n}$

Transpose of a matrix

$A\in R^{m\times n}$ and $A^T\in R^{n\times m}$

dot product.

$$\begin{array}{rl}⟨x,y⟩& =\sum _{i=1}^n{x}_i{y}_i\\ & =\sum _{i=1}^n{x}_i\cdot y_i\end{array}$$

linear combination of columns

Let $A\in R^{m\times n}$, $X\in R^n$, $Ax\in R^n$

Then $Ax={\sum }_{i=1}^n⟨a_i⟩x_i\in R^n$

euclidean norm

$L_2$ norm:

$$\Vert x{\Vert }_2=\sqrt{\sum _{i=1}^n{x}_i^2}=X^T\times X$$

L1 norm: $\Vert x{\Vert }_1={\sum }_{i=1}^n|x_i|$

$L_{\infty }$ norm: $\Vert x{\Vert }_{\infty }={\max }_i|x_i|$

p-norm: $\Vert x{\Vert }_p=({\sum }_{i=1}^n|x_i{|}^p{)}^{1/p}$

Comparison

$\Vert x{\Vert }_{\infty }\le \Vert x{\Vert }_2\le \Vert x{\Vert }_1$

One can prove this with Cauchy-Schwarz inequality

linear dependence of vectors

Given $\{x_1,x_2,\dots ,x_n\}\subseteq {\mathbb{R}}^d$ and ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_n\in \mathbb{R}$

$$\forall i\in [n],\forall \{a_1,a_2,\dots ,a_n\}\subseteq {\mathbb{R}}^{d}s.t.x_i\ne \sum _{j=1}^n{a}_j{x}_j$$