Loading history…
Loading history…
Linear algebra is the branch of mathematics concerned with vector spaces and the linear maps between them — the functions that respect vector addition and scaling. Worked concretely, it is the algebra of matrices and of systems of linear equations; worked abstractly, it is the theory of spaces of any dimension in which points can be added and rescaled.[1] Together with elementary algebra and abstract algebra, it is one of the main branches of algebra.
The subject owes its reach to a happy accident: linear problems are essentially the only large class of problems mathematics can solve completely, and an enormous range of questions either is linear or is attacked by linear approximation. The derivative in calculus is the best linear approximation to a function; geometry in n dimensions, the least-squares fitting of data, and the state spaces of quantum mechanics are all linear algebra at work.[2]
Systems of linear equations were being solved by elimination some two thousand years before the modern subject existed: chapter eight of the Chinese classic The Nine Chapters on the Mathematical Art arranges the coefficients of simultaneous equations on a counting board and reduces them column by column, in essence the method now taught as Gaussian elimination after Gauss.[3]
In Europe the theory grew out of determinants, which Leibniz studied in 1693 and Cramer used in 1750 to give a general formula for solving linear systems, now called Cramer's rule.[3] Matrices arrived surprisingly late: Sylvester coined the term matrix in 1850, and Cayley's 1858 memoir first treated matrices as objects of algebra in their own right, with their now-familiar addition, multiplication, and inverses.[4][3]
The abstract notion of a vector space emerged separately. Grassmann's Ausdehnungslehre (1844) developed a far-reaching theory of n-dimensional space that went largely unread in its time, and Peano distilled the modern axioms for a vector space in 1888.[note 1][3] In the twentieth century the axiomatic point of view took over, and the extension of linear algebra to infinite-dimensional spaces of functions — the work of Hilbert, Banach, and others — grew into the separate field of functional analysis.[5]
A vector space over a Field of scalars (most often the real or complex numbers) is a set of objects called vectors that can be added together and multiplied by scalars, subject to the familiar laws of arithmetic — associativity, commutativity of addition, distributivity, and the existence of a zero vector and of negatives.[6] The standard example is the space Rn of n-tuples of real numbers, added and scaled coordinate-wise, but the axioms are equally satisfied by spaces of polynomials, of matrices, or of functions — which is precisely what makes the theory portable.
A set of vectors is linearly independent if none of them can be written as a combination of the others, and it spans the space if every vector can be so written. A Basis is a set that does both, and the central bookkeeping fact of the subject is that any two bases of a vector space have the same number of elements, called its Dimension.[note 2][6] Once a basis is chosen, every vector is pinned down by its list of coordinates, and an abstract n-dimensional space becomes a copy of Rn.
A linear map (or linear transformation) T between vector spaces is a function compatible with the structure: T(v + w) = T(v) + T(w) and T(av) = a T(v) for all vectors v, w and scalars a. Rotations, reflections, projections, and scalings of space are linear; so are differentiation and integration on spaces of functions.[7]
After bases are chosen, a linear map between finite-dimensional spaces is recorded by a rectangular array of scalars — a matrix — and composing two maps corresponds to multiplying their matrices. This correspondence is the reason for the otherwise odd-looking row-by-column rule of matrix multiplication, and it lets every structural question about linear maps be computed with arrays of numbers.[7] The fundamental accounting principle relating a map to its matrix is the rank–nullity theorem: the dimension of the image plus the dimension of the kernel (the vectors sent to zero) equals the dimension of the domain.[6]
To each square matrix is attached a single scalar, its determinant, which measures the factor by which the corresponding map scales volume; the map is invertible exactly when its determinant is nonzero.[7]
The oldest face of the subject is the simultaneous solution of equations such as a1x + b1y + c1z = d1, each of which describes a plane (or, in higher dimensions, a hyperplane); solving the system means finding where they all intersect. In matrix language the whole system collapses to the single equation Ax = b, and Gaussian elimination solves it by systematically clearing coefficients below a staircase of pivots.[2]
The theory tells one exactly what to expect: a system has no solution, one solution, or an infinite family of solutions forming a translated subspace — a point, line, plane, and so on. When a system has no exact solution, as almost always happens with overdetermined experimental data, the method of least squares — introduced by Gauss and Legendre for astronomical orbit fitting — finds the closest approximate solution, and remains the workhorse of data fitting and statistics.[3][2]
An eigenvector of a linear map T is a nonzero vector whose direction the map leaves unchanged: T(v) = λv for some scalar λ, the corresponding eigenvalue. Eigenvectors are the axes along which a transformation acts by pure stretching, and when a space has a basis of them the map diagonalizes — its matrix becomes diagonal, and its powers and long-run behavior can be read off directly.[6]
This is the part of linear algebra other sciences lean on hardest. The spectral theorem guarantees that symmetric (self-adjoint) matrices diagonalize with mutually perpendicular eigenvectors, which underlies the principal axes of rotating bodies and of quadric surfaces, the normal modes of vibrating systems, principal component analysis in statistics, and the observable quantities of quantum mechanics.[6][2]
Adding an inner product — an abstraction of the dot product — to a vector space brings geometry into the theory: lengths, angles, and in particular orthogonality (perpendicularity) become meaningful. Any independent set of vectors can be straightened into an orthonormal basis by the Gram–Schmidt process, and orthogonal projection onto a subspace gives the geometric picture behind least squares.[6]
Complete infinite-dimensional inner product spaces are called Hilbert spaces; they are the setting in which Fourier analysis and quantum mechanics live, and their study belongs to functional analysis, linear algebra's infinite-dimensional continuation.[5]
Linear algebra is often described as the most widely applied part of mathematics after arithmetic. Within mathematics it supplies the language of analytic geometry and the linearization step in calculus and differential equations; multivariable calculus is largely the interplay of derivatives with linear maps. Outside it, matrices encode the geometry pipelines of computer graphics, the stiffness and circuit equations of engineering, the transition probabilities of Markov chains, the link structure of the web ranked by eigenvector methods, and the data arrays of statistics and machine learning, where matrix factorizations do the heavy lifting.[2][1] In physics, quantum mechanics is formulated outright as linear algebra in Hilbert space: states are vectors and observables are self-adjoint operators.[5]
Content available under CC BY-SA 4.0