Linear Algebra

You're viewing the source. Log in to edit.

'''Linear algebra''' is the branch of [[Mathematics|mathematics]] concerned with [[Vector space|vector space]]s and the [[Linear map|linear map]]s between them — the functions that respect vector addition and scaling. Worked concretely, it is the algebra of [[Matrix (mathematics)|matrices]] and of systems of linear equations; worked abstractly, it is the theory of spaces of any dimension in which points can be added and rescaled.<ref name="mathworld">"Linear Algebra." ''Wolfram MathWorld''. [https://mathworld.wolfram.com/LinearAlgebra.html mathworld.wolfram.com/LinearAlgebra.html]. Retrieved July 2026.</ref> Together with [[Elementary algebra|elementary algebra]] and [[Abstract algebra|abstract algebra]], it is one of the main branches of [[Algebra|algebra]].

The subject owes its reach to a happy accident: linear problems are essentially the only large class of problems mathematics can solve completely, and an enormous range of questions either is linear or is attacked by linear approximation. The derivative in calculus is the best linear approximation to a function; geometry in ''n'' dimensions, the least-squares fitting of data, and the state spaces of quantum mechanics are all linear algebra at work.<ref name="strang">Gilbert Strang, ''Introduction to Linear Algebra'' (5th ed., Wellesley-Cambridge Press, 2016).</ref>

== History ==

Systems of linear equations were being solved by elimination some two thousand years before the modern subject existed: chapter eight of the Chinese classic ''[[The Nine Chapters on the Mathematical Art]]'' arranges the coefficients of simultaneous equations on a counting board and reduces them column by column, in essence the method now taught as [[Gaussian elimination]] after [[Carl Friedrich Gauss|Gauss]].<ref name="boyer">Carl B. Boyer and Uta C. Merzbach, ''A History of Mathematics'' (3rd ed., Wiley, 2011).</ref>

In Europe the theory grew out of [[Determinant|determinant]]s, which [[Gottfried Wilhelm Leibniz|Leibniz]] studied in 1693 and [[Gabriel Cramer|Cramer]] used in 1750 to give a general formula for solving linear systems, now called [[Cramer's rule]].<ref name="boyer" /> Matrices arrived surprisingly late: [[James Joseph Sylvester|Sylvester]] coined the term ''matrix'' in 1850, and [[Arthur Cayley|Cayley]]'s 1858 memoir first treated matrices as objects of algebra in their own right, with their now-familiar addition, multiplication, and inverses.<ref name="cayley">Arthur Cayley, "A Memoir on the Theory of Matrices." ''Philosophical Transactions of the Royal Society of London'' 148 (1858).</ref><ref name="boyer" />

The abstract notion of a vector space emerged separately. [[Hermann Grassmann|Grassmann]]'s ''Ausdehnungslehre'' (1844) developed a far-reaching theory of ''n''-dimensional space that went largely unread in its time, and [[Giuseppe Peano|Peano]] distilled the modern axioms for a vector space in 1888.{{Note|Grassmann's book was so far ahead of its notation and audience that he reworked it entirely in 1862; wide recognition came only near the end of his life, and the axiomatization that made his ideas standard was Peano's.}}<ref name="boyer" /> In the twentieth century the axiomatic point of view took over, and the extension of linear algebra to infinite-dimensional spaces of functions — the work of [[David Hilbert|Hilbert]], [[Stefan Banach|Banach]], and others — grew into the separate field of [[Functional analysis|functional analysis]].<ref name="rudin">Walter Rudin, ''Functional Analysis'' (2nd ed., McGraw-Hill, 1991).</ref>

== Vector spaces ==

A [[Vector space|vector space]] over a [[Field (mathematics)|]] of scalars (most often the real or complex numbers) is a set of objects called vectors that can be added together and multiplied by scalars, subject to the familiar laws of arithmetic — associativity, commutativity of addition, distributivity, and the existence of a zero vector and of negatives.<ref name="axler">Sheldon Axler, ''Linear Algebra Done Right'' (3rd ed., Springer, 2015).</ref> The standard example is the space '''R'''<sup>''n''</sup> of ''n''-tuples of real numbers, added and scaled coordinate-wise, but the axioms are equally satisfied by spaces of polynomials, of matrices, or of functions — which is precisely what makes the theory portable.

A set of vectors is ''linearly independent'' if none of them can be written as a combination of the others, and it ''spans'' the space if every vector can be so written. A [[Basis (linear algebra)|]] is a set that does both, and the central bookkeeping fact of the subject is that any two bases of a vector space have the same number of elements, called its [[Dimension (vector space)|]].{{Note|In full generality — for spaces with no finite basis — the existence of a basis requires the axiom of choice. Finite-dimensional spaces, the home ground of linear algebra proper, need no such machinery.}}<ref name="axler" /> Once a basis is chosen, every vector is pinned down by its list of coordinates, and an abstract ''n''-dimensional space becomes a copy of '''R'''<sup>''n''</sup>.

== Linear maps and matrices ==

A [[Linear map|linear map]] (or linear transformation) ''T'' between vector spaces is a function compatible with the structure: ''T''('''v''' + '''w''') = ''T''('''v''') + ''T''('''w''') and ''T''(''a''<nowiki />'''v''') = ''a'' ''T''('''v''') for all vectors '''v''', '''w''' and scalars ''a''. Rotations, reflections, projections, and scalings of space are linear; so are differentiation and integration on spaces of functions.<ref name="hk">Kenneth Hoffman and Ray Kunze, ''Linear Algebra'' (2nd ed., Prentice-Hall, 1971).</ref>

After bases are chosen, a linear map between finite-dimensional spaces is recorded by a rectangular array of scalars — a [[Matrix (mathematics)|matrix]] — and composing two maps corresponds to multiplying their matrices. This correspondence is the reason for the otherwise odd-looking row-by-column rule of [[Matrix multiplication|matrix multiplication]], and it lets every structural question about linear maps be computed with arrays of numbers.<ref name="hk" /> The fundamental accounting principle relating a map to its matrix is the [[Rank–nullity theorem|rank–nullity theorem]]: the dimension of the image plus the dimension of the kernel (the vectors sent to zero) equals the dimension of the domain.<ref name="axler" />

To each square matrix is attached a single scalar, its [[Determinant|determinant]], which measures the factor by which the corresponding map scales volume; the map is invertible exactly when its determinant is nonzero.<ref name="hk" />

== Systems of linear equations ==

The oldest face of the subject is the simultaneous solution of equations such as ''a''<sub>1</sub>''x'' + ''b''<sub>1</sub>''y'' + ''c''<sub>1</sub>''z'' = ''d''<sub>1</sub>, each of which describes a plane (or, in higher dimensions, a hyperplane); solving the system means finding where they all intersect. In matrix language the whole system collapses to the single equation ''A''<nowiki />'''x''' = '''b''', and [[Gaussian elimination]] solves it by systematically clearing coefficients below a staircase of pivots.<ref name="strang" />

The theory tells one exactly what to expect: a system has no solution, one solution, or an infinite family of solutions forming a translated subspace — a point, line, plane, and so on. When a system has no exact solution, as almost always happens with overdetermined experimental data, the method of [[Least squares|least squares]] — introduced by Gauss and [[Adrien-Marie Legendre|Legendre]] for astronomical orbit fitting — finds the closest approximate solution, and remains the workhorse of data fitting and statistics.<ref name="boyer" /><ref name="strang" />

== Eigenvalues and eigenvectors ==

An [[Eigenvector|eigenvector]] of a linear map ''T'' is a nonzero vector whose direction the map leaves unchanged: ''T''('''v''') = ''λ''<nowiki />'''v''' for some scalar ''λ'', the corresponding [[Eigenvalue|eigenvalue]]. Eigenvectors are the axes along which a transformation acts by pure stretching, and when a space has a basis of them the map ''diagonalizes'' — its matrix becomes diagonal, and its powers and long-run behavior can be read off directly.<ref name="axler" />

This is the part of linear algebra other sciences lean on hardest. The [[Spectral theorem|spectral theorem]] guarantees that symmetric (self-adjoint) matrices diagonalize with mutually perpendicular eigenvectors, which underlies the principal axes of rotating bodies and of quadric surfaces, the normal modes of vibrating systems, principal component analysis in statistics, and the observable quantities of quantum mechanics.<ref name="axler" /><ref name="strang" />

== Inner product spaces ==

Adding an [[Inner product space|inner product]] — an abstraction of the dot product — to a vector space brings geometry into the theory: lengths, angles, and in particular ''orthogonality'' (perpendicularity) become meaningful. Any independent set of vectors can be straightened into an orthonormal basis by the [[Gram–Schmidt process]], and orthogonal projection onto a subspace gives the geometric picture behind least squares.<ref name="axler" />

Complete infinite-dimensional inner product spaces are called [[Hilbert space|Hilbert space]]s; they are the setting in which Fourier analysis and quantum mechanics live, and their study belongs to [[Functional analysis|functional analysis]], linear algebra's infinite-dimensional continuation.<ref name="rudin" />

== Applications ==

Linear algebra is often described as the most widely applied part of mathematics after arithmetic. Within mathematics it supplies the language of [[Analytic geometry|analytic geometry]] and the linearization step in calculus and differential equations; multivariable calculus is largely the interplay of derivatives with linear maps. Outside it, matrices encode the geometry pipelines of computer graphics, the stiffness and circuit equations of engineering, the transition probabilities of Markov chains, the link structure of the web ranked by eigenvector methods, and the data arrays of statistics and machine learning, where matrix factorizations do the heavy lifting.<ref name="strang" /><ref name="mathworld" /> In physics, quantum mechanics is formulated outright as linear algebra in Hilbert space: states are vectors and observables are self-adjoint operators.<ref name="rudin" />

== See also ==

* [[Algebra]]
* [[Matrix (mathematics)|]]
* [[Vector space]]
* [[Abstract algebra]]
* [[Functional analysis]]
* [[Geometry]]

Loading editor…

'''Linear algebra''' is the branch of [[Mathematics|mathematics]] concerned with [[Vector space|vector space]]s and the [[Linear map|linear map]]s between them — the functions that respect vector addition and scaling. Worked concretely, it is the algebra of [[Matrix (mathematics)|matrices]] and of systems of linear equations; worked abstractly, it is the theory of spaces of any dimension in which points can be added and rescaled.<ref name="mathworld">"Linear Algebra." ''Wolfram MathWorld''. [https://mathworld.wolfram.com/LinearAlgebra.html mathworld.wolfram.com/LinearAlgebra.html]. Retrieved July 2026.</ref> Together with [[Elementary algebra|elementary algebra]] and [[Abstract algebra|abstract algebra]], it is one of the main branches of [[Algebra|algebra]]. The subject owes its reach to a happy accident: linear problems are essentially the only large class of problems mathematics can solve completely, and an enormous range of questions either is linear or is attacked by linear approximation. The derivative in calculus is the best linear approximation to a function; geometry in ''n'' dimensions, the least-squares fitting of data, and the state spaces of quantum mechanics are all linear algebra at work.<ref name="strang">Gilbert Strang, ''Introduction to Linear Algebra'' (5th ed., Wellesley-Cambridge Press, 2016).</ref> == History == Systems of linear equations were being solved by elimination some two thousand years before the modern subject existed: chapter eight of the Chinese classic ''[[The Nine Chapters on the Mathematical Art]]'' arranges the coefficients of simultaneous equations on a counting board and reduces them column by column, in essence the method now taught as [[Gaussian elimination]] after [[Carl Friedrich Gauss|Gauss]].<ref name="boyer">Carl B. Boyer and Uta C. Merzbach, ''A History of Mathematics'' (3rd ed., Wiley, 2011).</ref> In Europe the theory grew out of [[Determinant|determinant]]s, which [[Gottfried Wilhelm Leibniz|Leibniz]] studied in 1693 and [[Gabriel Cramer|Cramer]] used in 1750 to give a general formula for solving linear systems, now called [[Cramer's rule]].<ref name="boyer" /> Matrices arrived surprisingly late: [[James Joseph Sylvester|Sylvester]] coined the term ''matrix'' in 1850, and [[Arthur Cayley|Cayley]]'s 1858 memoir first treated matrices as objects of algebra in their own right, with their now-familiar addition, multiplication, and inverses.<ref name="cayley">Arthur Cayley, "A Memoir on the Theory of Matrices." ''Philosophical Transactions of the Royal Society of London'' 148 (1858).</ref><ref name="boyer" /> The abstract notion of a vector space emerged separately. [[Hermann Grassmann|Grassmann]]'s ''Ausdehnungslehre'' (1844) developed a far-reaching theory of ''n''-dimensional space that went largely unread in its time, and [[Giuseppe Peano|Peano]] distilled the modern axioms for a vector space in 1888.{{Note|Grassmann's book was so far ahead of its notation and audience that he reworked it entirely in 1862; wide recognition came only near the end of his life, and the axiomatization that made his ideas standard was Peano's.}}<ref name="boyer" /> In the twentieth century the axiomatic point of view took over, and the extension of linear algebra to infinite-dimensional spaces of functions — the work of [[David Hilbert|Hilbert]], [[Stefan Banach|Banach]], and others — grew into the separate field of [[Functional analysis|functional analysis]].<ref name="rudin">Walter Rudin, ''Functional Analysis'' (2nd ed., McGraw-Hill, 1991).</ref> == Vector spaces == A [[Vector space|vector space]] over a [[Field (mathematics)|]] of scalars (most often the real or complex numbers) is a set of objects called vectors that can be added together and multiplied by scalars, subject to the familiar laws of arithmetic — associativity, commutativity of addition, distributivity, and the existence of a zero vector and of negatives.<ref name="axler">Sheldon Axler, ''Linear Algebra Done Right'' (3rd ed., Springer, 2015).</ref> The standard example is the space '''R'''''n'' of ''n''-tuples of real numbers, added and scaled coordinate-wise, but the axioms are equally satisfied by spaces of polynomials, of matrices, or of functions — which is precisely what makes the theory portable. A set of vectors is ''linearly independent'' if none of them can be written as a combination of the others, and it ''spans'' the space if every vector can be so written. A [[Basis (linear algebra)|]] is a set that does both, and the central bookkeeping fact of the subject is that any two bases of a vector space have the same number of elements, called its [[Dimension (vector space)|]].{{Note|In full generality — for spaces with no finite basis — the existence of a basis requires the axiom of choice. Finite-dimensional spaces, the home ground of linear algebra proper, need no such machinery.}}<ref name="axler" /> Once a basis is chosen, every vector is pinned down by its list of coordinates, and an abstract ''n''-dimensional space becomes a copy of '''R'''''n''. == Linear maps and matrices == A [[Linear map|linear map]] (or linear transformation) ''T'' between vector spaces is a function compatible with the structure: ''T''('''v''' + '''w''') = ''T''('''v''') + ''T''('''w''') and ''T''(''a''<nowiki />'''v''') = ''a'' ''T''('''v''') for all vectors '''v''', '''w''' and scalars ''a''. Rotations, reflections, projections, and scalings of space are linear; so are differentiation and integration on spaces of functions.<ref name="hk">Kenneth Hoffman and Ray Kunze, ''Linear Algebra'' (2nd ed., Prentice-Hall, 1971).</ref> After bases are chosen, a linear map between finite-dimensional spaces is recorded by a rectangular array of scalars — a [[Matrix (mathematics)|matrix]] — and composing two maps corresponds to multiplying their matrices. This correspondence is the reason for the otherwise odd-looking row-by-column rule of [[Matrix multiplication|matrix multiplication]], and it lets every structural question about linear maps be computed with arrays of numbers.<ref name="hk" /> The fundamental accounting principle relating a map to its matrix is the [[Rank–nullity theorem|rank–nullity theorem]]: the dimension of the image plus the dimension of the kernel (the vectors sent to zero) equals the dimension of the domain.<ref name="axler" /> To each square matrix is attached a single scalar, its [[Determinant|determinant]], which measures the factor by which the corresponding map scales volume; the map is invertible exactly when its determinant is nonzero.<ref name="hk" /> == Systems of linear equations == The oldest face of the subject is the simultaneous solution of equations such as ''a''1''x'' + ''b''1''y'' + ''c''1''z'' = ''d''1, each of which describes a plane (or, in higher dimensions, a hyperplane); solving the system means finding where they all intersect. In matrix language the whole system collapses to the single equation ''A''<nowiki />'''x''' = '''b''', and [[Gaussian elimination]] solves it by systematically clearing coefficients below a staircase of pivots.<ref name="strang" /> The theory tells one exactly what to expect: a system has no solution, one solution, or an infinite family of solutions forming a translated subspace — a point, line, plane, and so on. When a system has no exact solution, as almost always happens with overdetermined experimental data, the method of [[Least squares|least squares]] — introduced by Gauss and [[Adrien-Marie Legendre|Legendre]] for astronomical orbit fitting — finds the closest approximate solution, and remains the workhorse of data fitting and statistics.<ref name="boyer" /><ref name="strang" /> == Eigenvalues and eigenvectors == An [[Eigenvector|eigenvector]] of a linear map ''T'' is a nonzero vector whose direction the map leaves unchanged: ''T''('''v''') = ''λ''<nowiki />'''v''' for some scalar ''λ'', the corresponding [[Eigenvalue|eigenvalue]]. Eigenvectors are the axes along which a transformation acts by pure stretching, and when a space has a basis of them the map ''diagonalizes'' — its matrix becomes diagonal, and its powers and long-run behavior can be read off directly.<ref name="axler" /> This is the part of linear algebra other sciences lean on hardest. The [[Spectral theorem|spectral theorem]] guarantees that symmetric (self-adjoint) matrices diagonalize with mutually perpendicular eigenvectors, which underlies the principal axes of rotating bodies and of quadric surfaces, the normal modes of vibrating systems, principal component analysis in statistics, and the observable quantities of quantum mechanics.<ref name="axler" /><ref name="strang" /> == Inner product spaces == Adding an [[Inner product space|inner product]] — an abstraction of the dot product — to a vector space brings geometry into the theory: lengths, angles, and in particular ''orthogonality'' (perpendicularity) become meaningful. Any independent set of vectors can be straightened into an orthonormal basis by the [[Gram–Schmidt process]], and orthogonal projection onto a subspace gives the geometric picture behind least squares.<ref name="axler" /> Complete infinite-dimensional inner product spaces are called [[Hilbert space|Hilbert space]]s; they are the setting in which Fourier analysis and quantum mechanics live, and their study belongs to [[Functional analysis|functional analysis]], linear algebra's infinite-dimensional continuation.<ref name="rudin" /> == Applications == Linear algebra is often described as the most widely applied part of mathematics after arithmetic. Within mathematics it supplies the language of [[Analytic geometry|analytic geometry]] and the linearization step in calculus and differential equations; multivariable calculus is largely the interplay of derivatives with linear maps. Outside it, matrices encode the geometry pipelines of computer graphics, the stiffness and circuit equations of engineering, the transition probabilities of Markov chains, the link structure of the web ranked by eigenvector methods, and the data arrays of statistics and machine learning, where matrix factorizations do the heavy lifting.<ref name="strang" /><ref name="mathworld" /> In physics, quantum mechanics is formulated outright as linear algebra in Hilbert space: states are vectors and observables are self-adjoint operators.<ref name="rudin" /> == See also == * [[Algebra]] * [[Matrix (mathematics)|]] * [[Vector space]] * [[Abstract algebra]] * [[Functional analysis]] * [[Geometry]]