Least Squares Fitting

Least squares fitting is a mathematical procedure for finding the best-fitting curve to a given set of points by minimizing the sum of the squares of the offsets (“the residuals”) of the points from the curve. Least squares means that the overall solution minimizes the sum of the squares of the errors made in the results of every single equation.

The function least.squares() in the animation package has provided an illustration of least squares in univariate linear regression.

let’s assume that the relationship between x andy is a linear one.

Let $y=ax+b$ be the eauation of the best fit line to the data. We need to determine the valuses of both the slop $a$ and intercept $b$ . Assuming each data point carries eauql weight, each $y_{i}$ point has exactly the same actual error asscociated with it, we find $a$ and $b$ by minimizing the sum of the squares of the deviations of the actual values of $y_{i}$ from the line’s calcualted value of y.

The formulas for $a$ and $b$ are:

$a=SLOPE=\frac{n\sum{x_{i}y_{i}}-\sum {x_{i}}\sum{y_{i}}}{n\sum{x_{i}^{2}}-(\sum{x_{i}})^2}$ $b=INTERCEPT=\frac{\sum{x_{i}^{2}}\sum{y_{i}}-\sum{x_{i}}\sum{y_{i}}}{n\sum{x_{i}^{2}}-(\sum{x_{i}})^2}$

Here is the simple demonstration of the meaning of least squares in univariate linear regression.

With slope changing

library(animation)
ani.options(interval = 0.3)
par(mar = c(4, 4, 0.5, 0.1), mgp = c(2, 0.5, 0), tcl = -0.3)
## slope changing
least.squares()

With intercept changing

library(animation)
ani.options(interval = 0.3)
par(mar = c(4, 4, 0.5, 0.1), mgp = c(2, 0.5, 0), tcl = -0.3)
# slope changing
least.squares(ani.type = "intercept")

← Previous Archive Next →

Published

08 May 2013

Least Squares Fitting

With slope changing

With intercept changing

Published

Tags