See also slides for curve fitting, regression, colab link

Transclude of thoughts/university/twenty-four-twenty-five/sfwr-4ml3/ols_and_kls.py

curve fitting.

how do we fit a distribution of data over a curve?

Given a set of data points

  • (or )

ols.

Ordinary Least Squares (OLS)

Let be the prediction of a model , is the error, minimize

In the case of 1-D ordinary least square, the problems equates find to minimize

Optimal solution

where , , ,

Hyperplane equation

where is the intercept (bias)

Homogenous hyperplane:

minimize

Thus we can find

Example:

With and $$ X^{’}{n \times 3} = \begin{pmatrix} x^{1}{1} & x^{1}{2} & 1 \ x^{2}{1} & x^{2}{2} & 1 \ x^{3}{1} & x^{3}_{2} & 1 \end{pmatrix}

With $$ W = \begin{pmatrix} w_1 \\ w_2 \end{pmatrix} $$ and $$ W^{'} = \begin{pmatrix} w_1 \\ w_2 \\ w_0 \end{pmatrix} $$ thus

X^{’} \times W = \begin{pmatrix} w_0 + \sum{w_i \times x_i^{1}} \ \vdots \ w_0 + \sum{w_i \times x_i^{n}} \end{pmatrix}