If A is square and invertible, we want to have the solution x = A−1b.
If A is underconstrained, we want the entire set of solutions.
If A is overconstrained, we could simply give up. But this case arises a lot in practice, so instead we will ask for the least-squares solution. In other words, we want that ¯x which minimizes the error kA¯x − bk. Geometrically, A¯x is the point in the column space of A closest to b.
Three situations arise regarding the basic goal