How to Find an Estimate of the Least Squares
- 1). Write equations in the form y = mx + b for all your data points. Given the points (1, 1), (2, 4), (3, 2) and (4,3), you would write the following equations:
1 = m + b
4 = 2m + b
2 = 3m + b
3 = 4m + b - 2). Set up matrix A by using the coefficients from the right side of the equations. Given equations 1 = m + b, 4 = 2m + b, 2 = 3m + b and 3 = 4m + b, you would set up the matrix A as follows:
[
1, 1
2, 1
3, 1
4, 1
] - 3). Establish vector V by using the variables m and b such that they are the least squares estimate, i.e., m*, b*.
[
m*
b*
] - 4). Build vector B by using the corresponding y coordinates. Given the points (1, 1), (2, 4), (3, 2) and (4,3), vector B would look as follows:
[
1
4
2
3
] - 5). Write the transpose of matrix A. Remember that the transpose of any matrix M is a matrix in which the rows of M are the columns of Mt, and the columns of M are the rows of Mt. Given the example, the transpose of matrix A is:
[
1, 2, 3, 4
1, 1, 1, 1
] - 6). Calculate AtA by multiplying the transpose of matrix A by matrix A using the dot product method. Given the example, multiplying the 2x4 transpose by its 4x2 original would lead to the following 2x2 matrix:
[
1(1) + 2(2) + 3(3) + 4(4), 1(1) + 2(1) + 3(1) + 4(1)
1(1) + 1(2) + 1(3) + 1(4), 1(1) + 1(1) + 1(1) + 1(1)
]
This would then be simplified into the following:
[
30, 10
10, 5
] - 7). Calculate AtB by multiplying the transpose of matrix A by vector B using the dot product method. Given the example, multiplying the 2x4 transpose by the 4x1 vector would lead to the following 2x1 matrix:
[
1(1) + 2(4) + 3(2) + 4(3)
1(1) + 1(4) + 1(2) + 1(3)
]
This would then be simplified into the following:
[
27
9
] - 8). Write equations in the form y = mx + b by using the values of AtT, AtB and vector V. Given the example you would write the following equations:
30m + 10b = 27
10m + 5b = 9 - 9). Solve for m by using elimination. Given the equations 30m + 10b = 27 and 10m + 5b = 9, for instance, you would multiply the second equation by -2 and then add it to the first equation:
30m + 10b = 27
-20m - 10b = -18
----------------------------
10m = 9
m = 9/10 - 10
Solve for b by using substitution. Given that m = 9/10, you can substitute 9/10 into the equation 30m + 10b = 27:
30(9/10) + 10b = 27
18 + 10b = 27
10b = 9
b = 9/10 - 11
Find the estimate of the least squares by plugging the values for m and b into the standard equation of a line. Given the example, the estimate of the least squares for the points (1, 1), (2, 4), (3, 2) and (4,3) is the line y = 9x/10 + 9/10.
Source...