Engineering/Mathematics/Preliminaries
Linear Algebra Theorem Map — Foundations and Core Equivalences
Linear algebra is not a list of tricks but a directed graph: every major theorem is proved from a small number of earlier ones, and almost every path in that graph passes through the pivot count . This page charts the graph layer by layer, naming what each theorem establishes and what it rests on.
- Core level
- Stream: reference
- Reading time 18 min
- Ref KVS-ENG-MATH-0123
- Taxonomy
- Engineering / Mathematics
- Root results
- Existence and uniqueness of reduced row-echelon form
- Central invariant
- The pivot count , which becomes the rank
- Structure
- Six dependency layers, matrices at the base, transformations at the top
- Hub theorems
- Rank plus nullity, the Goldilocks theorem, the fundamental theorem of matrix representation
- Standing hypotheses
- Complex scalars and finite dimension above layer two
Overview
A first pass through linear algebra can feel like an unordered inventory of results: row reduction, span, independence, bases, dimension, rank, determinants, eigenvalues, transformations. The subject is in fact tightly ordered. Each theorem is proved from a short list of earlier theorems, so the whole body of results forms a directed acyclic graph in which the edges are proof dependencies. Understanding that graph is what separates a reader who can recall a statement from an engineer who can decide, on sight, which result settles the question in front of them.
The graph has a single narrow base. Everything downstream begins with three reversible row operations, the fact that they do not disturb a solution set, and the existence and uniqueness of the reduced row-echelon form they produce. From that root emerges one integer — , the number of non-zero rows of the reduced form — and that integer propagates through the entire subject. It counts the pivot columns, it fixes the number of free variables at , it becomes the rank of the matrix, it gives the dimension of the column space and the row space, it decides whether a square matrix is invertible, and it reappears as the rank of a linear transformation. Most of the theorems in the middle of the subject are statements about what controls.
Above that core the graph branches. One branch abstracts from column vectors to arbitrary vector spaces and produces span, independence, bases and dimension, culminating in a counting theorem that makes set size alone decide two of the three defining properties of a basis. A second branch attaches four subspaces to every matrix and relates their dimensions. A third introduces the determinant as a single scalar certificate and then the eigenvalues as the numbers that describe how a square matrix acts on its own invariant directions. A fourth replaces matrices with linear transformations between abstract spaces, and then closes the loop by proving that every finite-dimensional transformation is a matrix once bases are chosen.
The practical value of holding this structure in mind is economy. Questions have a natural depth. Whether a truss is statically indeterminate, whether a sensor set determines a state, whether a set of predictors is redundant — these are all layer-two questions answered by alone, at cubic cost and with no eigenvalue computation. Whether a system is stable, whether a matrix can be diagonalised, whether two matrices represent the same operator — these live several layers higher and genuinely need the spectral machinery. Reaching for the wrong layer is the most common source of unnecessary work and of numerically fragile code.
Definition
Reduced Row-Echelon Form
RREFThe canonical representative of a matrix under row equivalence. A matrix is in reduced row-echelon form when every zero row lies below every non-zero row, the leftmost non-zero entry of each non-zero row equals , each such leading one lies strictly to the right of the leading one above it, and each leading one is the only non-zero entry in its column. Every matrix is row-equivalent to exactly one matrix in this form.
This is the root of the dependency graph. Existence is proved by an algorithm; uniqueness is proved separately and is what licenses the definite article in phrases such as the rank of a matrix.
Rank of a Matrix
ROMThe dimension of the column space of , written or . Operationally it equals , the number of non-zero rows of the reduced row-echelon form of , equivalently the number of pivot columns. The companion quantity is the nullity , the dimension of the null space, which equals .
Rank is the single quantity with the highest degree in the dependency graph: results in every layer above the second either compute it, constrain it or consume it.
Multiple Equivalences
MEA theorem whose statement is a list of assertions declared mutually equivalent — any two may be joined by if and only if. A list of statements therefore packages separate theorems. Such a theorem is normally proved as a cycle of implications , which costs proofs rather than .
The equivalences for nonsingular matrices are the largest such theorem in the subject and act as a vertical spine through the dependency chart, gaining a new statement in almost every layer.
Theorem Dependency Graph
The directed graph whose vertices are the definitions and theorems of the subject and whose edges run from a result to every later result whose proof invokes it. Because no proof may use a result that depends on it, the graph is acyclic and admits a topological ordering — which is precisely a valid order in which the subject can be developed. Layers in this page are levels of that ordering, and a hub is a vertex of unusually high out-degree.
The graph is not unique: an alternative development can reorder results, prove eigenvalues without determinants, or take invertibility rather than a trivial null space as the primitive notion. What is invariant is which results genuinely require which hypotheses.
Concepts
Layer 0 — row operations and the canonical form
The base of the chart contains four results and nothing else depends on anything outside them. They establish that row manipulation is safe and that it has a well-defined destination.
- Equation Operations Preserve Solution Sets (EOPSS) — establishes that swapping two equations, multiplying an equation by a non-zero scalar and adding a multiple of one equation to another leave the solution set untouched. Rests on: the definition of a solution set alone. This is the only genuinely primitive theorem in the subject.
- Row-Equivalent Matrices represent Equivalent Systems (REMES) — establishes that the same three operations applied to the rows of an augmented matrix preserve the solution set. Rests on: EOPSS plus the correspondence between a system and its augmented matrix.
- Row-Equivalent Matrix in Echelon Form (REMEF) — establishes existence: every matrix is row-equivalent to a matrix in reduced row-echelon form. Rests on: the row-reducing algorithm, which is a constructive proof.
- Reduced Row-Echelon Form is Unique (RREFU) — establishes uniqueness: if two matrices in reduced row-echelon form are both row-equivalent to , they are equal. Rests on: an induction on pivot positions and the correspondence between rows and linear combinations. It depends on nothing above it, which is why it can be quoted freely everywhere else.
Together these two halves make reduced row-echelon form a canonical form. Existence gives an algorithm; uniqueness makes the algorithm's output a function of the input rather than of the route taken. Every quantity read from a reduced form — the pivot set, the rank, a null space basis — is well defined only because of RREFU, and every later theorem that says “let be the reduced row-echelon form of ” silently cites it.
Layer 1 — the pivot count governs every linear system
One integer emerges from the base layer and organises the next. Write for the number of non-zero rows of the reduced form of an augmented matrix with variables. Every question about a single linear system is answered by , and one bit indicating whether the final column holds a pivot.
- Recognising Consistency of a Linear System (RCLS) — establishes that a system is inconsistent exactly when the reduced augmented matrix has a leading one in column . Rests on: REMES and RREFU. This is the decision procedure the whole first chapter is built to deliver.
- Consistent Systems, and (CSRN) — establishes that a consistent system has exactly free variables, so the solution set is a single point when and infinite otherwise. Rests on: RCLS.
- Possible Solution Sets for Linear Systems (PSSLS) — establishes the trichotomy: no solution, exactly one, or infinitely many. Rests on: CSRN. There is no fourth possibility, and no linear system has, say, exactly seven solutions.
- Consistent, More Variables than Equations (CMVEI) — establishes that a consistent system with has infinitely many solutions. Rests on: CSRN and the inequality . Notably it needs no computation at all, only the shape of the system.
- Homogeneous Systems are Consistent (HSC) and Homogeneous, More Variables than Equations (HMVEI) — establish that always has the zero solution, and has others whenever . Rests on: CMVEI. This pair is the engine behind every later independence argument.
The step that makes this layer permanent is the reinterpretation of as an intrinsic quantity. Computing Rank and Nullity (CRN) establishes and , converting a count of rows in an algorithm's output into a pair of dimensions. From that point on, results can be stated about rank without any reference to row reduction, and row reduction survives only as the means of computing it.
Layer 2 — span, independence, bases and dimension
The second branch abstracts away from systems. A set of vectors, rather than a matrix, becomes the object of study, and three properties of a set — spanning, independence, and being a basis — are related to one another and eventually to a single integer.
- Solutions to Linear Systems are Linear Combinations (SLSLC) — establishes that is solvable exactly when is a linear combination of the columns of . Rests on: the definition of the matrix-vector product. This single theorem is the bridge between the system layer and the vector layer, and almost every result above it uses it at least once.
- Vector Form of Solutions to Linear Systems (VFSLS) and Particular Solution Plus Homogeneous Solutions (PSPHS) — establish that a solution set is a single particular solution translated by the null space. Rests on: SLSLC and CSRN.
- Linearly Independent Vectors and Homogeneous Systems (LIVHS) — establishes that a set of column vectors is independent exactly when the homogeneous system with those columns has only the trivial solution. Rests on: SLSLC. Independence becomes a computation.
- More Vectors than Size implies Linear Dependence (MVSLD) — establishes that any set of more than vectors in is dependent. Rests on: LIVHS and HMVEI. This is the first purely numerical constraint on a set of vectors.
- Basis for Null Spaces (BNS) and Basis of a Span (BS) — establish that row reduction produces an independent spanning set, not merely a spanning set. Rests on: SSNS, DLDS and the pivot structure of the reduced form.
- Spanning Sets and Linear Dependence (SSLD) — establishes that if vectors span a space then any or more vectors in it are dependent. Rests on: MVSLD lifted to an abstract space. Almost every dimension result traces back here.
- Bases have Identical Sizes (BIS) — establishes that two bases of the same space have the same number of vectors, which is what makes dimension definable at all. Rests on: SSLD applied twice, once in each direction.
The layer terminates in a counting theorem of unusual leverage. The Goldilocks theorem (G) states that in a space of dimension , a set of vectors is dependent, a set of vectors cannot span, and a set of exactly vectors is a basis as soon as either independence or spanning is verified. It rests on SSLD, BIS and the extension theorem ELIS. Its practical payoff is large: verifying independence is one homogeneous system, whereas verifying spanning directly would require solving a system for an arbitrary target vector.
Layer 3 — the four subspaces, rank and rank-nullity
The third layer attaches four subspaces to every matrix and computes their dimensions. It is where the abstract vocabulary of layer two is reconnected to the concrete arithmetic of layer one.
- Column Spaces and Consistent Systems (CSCS) — establishes if and only if is consistent. Rests on: SLSLC. The column space is exactly the set of achievable right-hand sides.
- Basis of the Column Space (BCS) — establishes that the columns of in the pivot positions of its reduced form are a basis for . Rests on: BS. Note the subtlety: the original columns form the basis, not the reduced ones.
- Row-Equivalent Matrices have equal Row Spaces (REMRS) and Basis for the Row Space (BRS) — establish that the non-zero rows of the reduced form are a basis for the row space. Rests on: the fact that each row operation replaces a row by a linear combination of rows.
- Column Space, Row Space, Transpose (CSRST) — establishes , so the two spaces are the same object viewed twice. Rests on: the involution property of the transpose.
- Properties of Extended Echelon Form (PEEF) and Four Subsets (FS) — establish that reducing yields two submatrices from which all four subspaces can be read: the null space and row space from one, the column space and left null space from the other. Rests on: BNS, BRS and the record of the transforming matrix.
- Rank Plus Nullity is Columns (RPNC) — establishes . Rests on: CRN alone, since . The proof is trivial; the content sits entirely in CRN, which is in turn licensed by RREFU.
- Rank of a Matrix is the Rank of the Transpose (RMRT) — establishes , that is, row rank equals column rank. Rests on: the Goldilocks theorem and BRS. This is the least obvious result in the layer and the one that most clearly repays the abstraction of layer two: no direct manipulation of entries proves it easily.
The layer also supplies the tool used constantly to prove two subspaces equal without exhibiting a bijection: Equal Dimensions Yields Equal Subspaces (EDYES) says that if and then . It converts a statement about integers into a statement about sets, and it rests on the Goldilocks theorem.
Layer 4 — determinants and eigenvalues
The fourth layer changes the question. Instead of asking what a matrix does to arbitrary vectors, it asks which directions the matrix leaves invariant. Two independent sub-branches feed it: the determinant, a single scalar certificate, and the eigenvalue theory that the determinant makes computable.
- Determinant Expansion about Rows and Columns (DERC) — establishes that cofactor expansion along any row or column gives the same value. Rests on: induction on size. It is what makes the determinant well defined rather than a choice.
- Determinant Respects Matrix Multiplication (DRMM) and Determinant of the Transpose (DT) — establish and . Rests on: elementary matrix decompositions.
- Singular Matrices have Zero Determinants (SMZD) — establishes that is singular if and only if , joining the determinant branch to the main trunk. Rests on: the behaviour of the determinant under row operations.
- Every Matrix Has an Eigenvalue (ESMHE) — establishes that every square matrix over has at least one eigenvalue. Rests on: the fundamental theorem of algebra, imported from outside linear algebra, together with a dependence argument on the powers . This is where the choice of scalar field becomes load-bearing: over the statement is false.
- Eigenvalues are Roots of the Characteristic Polynomial (EMRCP) and Eigenspace of a Matrix is a Null Space (EMNS) — establish is an eigenvalue, and . Rests on: SMZD and the equivalences for nonsingular matrices. These two theorems reduce the eigenvector problem to a null space problem — a layer-one computation — which is why the whole spectral theory is computable at all.
- Multiplicities of an Eigenvalue (ME) — establishes , the geometric multiplicity never exceeding the algebraic. Rests on: similarity invariance of the characteristic polynomial.
- Diagonalisation Characterisation (DC), Diagonalizable Matrices have Large Eigenspaces (DMLE) and Distinct Eigenvalues implies Diagonalizable (DED) — establish, respectively, that is diagonalisable exactly when has a basis of eigenvectors of ; exactly when for every eigenvalue; and that distinct eigenvalues suffice. Rests on: EDELI, ME and the Goldilocks theorem.
The dependency worth noticing is that the eigenvalue branch consumes the layers below it wholesale. An eigenspace is a null space, so computing one is row reduction; deciding diagonalisability is a dimension count; and the proof that eigenvectors for distinct eigenvalues are independent is a relation-of-linear-dependence argument from layer two. Nothing in the eigenvalue layer is self-contained.
Layer 5 — transformations, isomorphism and representation
The top of the chart replaces matrices with functions between vector spaces and then proves that, in finite dimension, nothing has been gained or lost. This is the closure that makes the subject a single theory rather than two.
- Matrices Build Linear Transformations (MBLT) and Matrix of a Linear Transformation (MLTCV) — establish that is always linear, and that every linear arises this way from a unique . Rests on: the distributive properties of the matrix-vector product and the standard unit vectors.
- Kernel of an Injective Linear Transformation (KILT) and Range of a Surjective Linear Transformation (RSLT) — establish that injectivity is a trivial kernel and surjectivity is a full range. Rests on: LTTZZ. These are the transformation-level analogues of a trivial null space and a full column space.
- Invertible Linear Transformations are Injective and Surjective (ILTIS) — establishes the expected characterisation of invertibility. Rests on: KILT, RSLT and the definition of a two-sided inverse function.
- Rank Plus Nullity is Domain Dimension (RPNDD) — establishes for . Rests on: a basis-extension construction and the Goldilocks theorem. It is the exact analogue of RPNC one layer up, and the two are provably the same statement once a matrix representation is chosen.
- Characterisation of Finite Dimensional Vector Spaces (CFDVS) and Isomorphism of Finite Dimensional Vector Spaces (IFDVS) — establish that every space of dimension is isomorphic to , and that two finite-dimensional spaces are isomorphic exactly when their dimensions agree. Rests on: VRILT, the proof that vector representation relative to a basis is an invertible linear transformation. Dimension is therefore a complete invariant.
- Fundamental Theorem of Matrix Representation (FTMR) — establishes : coordinatise the input, multiply by a matrix, and the result is the coordinatisation of the output. Rests on: LTDB and VRLT. Every question about an abstract transformation becomes a question about a matrix, and the whole chart below becomes available.
- Kernel and Null Space Isomorphism (KNSI), Range and Column Space Isomorphism (RCSI), Invertible Matrix Representations (IMR) and Similarity and Change of Basis (SCB) — establish the dictionary entry by entry: kernel to null space, range to column space, invertible transformation to invertible matrix, change of basis to similarity. Rests on: FTMR throughout.
Reading the chart by out-degree identifies the hubs. Uniqueness of the reduced form and the theorem converting solutions into linear combinations dominate the lower half; the Goldilocks theorem and rank plus nullity dominate the middle; the fundamental theorem of matrix representation dominates the top. The equivalences for nonsingular matrices run vertically through all of them, absorbing one new statement per layer, and are best read as a summary of the whole chart restricted to square matrices.
Locating a question on the chart
Equations
Canonical form at the root of the chart
EQ-MAP-01Row equivalence is decided by comparing two specific matrices. Everything downstream that speaks of “the” reduced form, “the” pivot columns or “the” rank depends on the uniqueness half of this statement.
The pivot count decides the solution set
EQ-MAP-02The complete classification of solution sets for a system in variables, where is the pivot column set of the reduced augmented matrix. The degrees of freedom are .
Rank plus nullity is columns
EQ-MAP-03The conservation law of the middle layers. Every column of either contributes a dimension to the column space or a degree of freedom to the null space, never both and never neither.
The Goldilocks theorem
EQ-MAP-04Set size alone decides two of the three cases outright, and in the third case reduces two verifications to one. This is the highest-leverage counting result in the subject.
Row rank equals column rank
EQ-MAP-05The number of independent rows equals the number of independent columns for every matrix of every shape. Elementary to state, and reachable only after dimension theory is in place.
Characteristic polynomial and its degree
EQ-MAP-06The entry point to layer four. Its degree bounds the number of distinct eigenvalues by , and over the fundamental theorem of algebra guarantees at least one root.
Fundamental theorem of matrix representation
EQ-MAP-07Applying a linear transformation and then coordinatising gives the same result as coordinatising and then multiplying by a matrix. This identity closes the loop between the top and bottom of the chart.
Variable Definitions
| Symbol | Name | Meaning | Domain / type |
|---|---|---|---|
| Matrix | The matrix under study, with complex entries | m x n matrix | |
| Row count | Number of equations, or the size of the codomain | positive integer | |
| Column count | Number of unknowns, or the size of the domain | positive integer | |
| Pivot count and rank | Non-zero rows of the reduced form; becomes and | 0 to min(m,n) | |
| Pivot column set | Indices of the columns carrying leading ones; determines every structural reading | subset of 1..n | |
| Dimension of a vector space | Common size of every basis of , written | non-negative integer | |
| Eigenvalue | Scalar with for some non-zero | complex number | |
| Algebraic multiplicity | Multiplicity of as a root of | 1 to n | |
| Geometric multiplicity | , never larger than | 1 to n | |
| Matrix representation | Matrix of a linear transformation relative to a domain basis and a codomain basis | matrix over C |
Worked Numerical Example
Problem statement
Four instruments observe a six-state process, giving a measurement matrix. Trace one reduction through the chart and extract, layer by layer, every conclusion the structure supports — and identify the question the chart says cannot be asked at all.
Assemble the measurement matrix
Each row is one instrument's sensitivity to the six state variables . The matrix is the object; the chart is the map for interrogating it.
Layer 0: reduce to the canonical form
Gauss-Jordan elimination gives the reduced row-echelon form. Uniqueness guarantees that any other sequence of row operations reaches the same destination, so every number extracted below is a property of rather than of the route.
Layer 1: read , the pivot set and the free variables
Three non-zero rows give , with pivot columns and free columns . For any consistent right-hand side there are degrees of freedom, so no measurement vector can ever determine the state uniquely. That verdict required nothing above layer one.
Layer 2: a basis for the unobservable subspace
Setting each free variable to one in turn and reading the dependent variables off the reduced rows gives three vectors. They are independent by construction — each carries a in a position where the others carry — so they are a basis, not merely a spanning set. Any state differing by a vector in this span produces identical readings on all four instruments.
Layer 3: column space, row space and the transpose
The pivot columns index the original columns of that form a basis for the column space, so . The three non-zero rows of the reduced form are a basis for the row space, so as well — an instance of row rank equalling column rank. Rank plus nullity checks out: .
Layer 3: the left null space and the consistency criterion
The left null space has dimension . Its single basis vector records the linear dependency among the instrument rows: row four is the sum of rows two and three. Because the column space is exactly the null space of the left null space basis, this one vector is a complete consistency test — a reading vector is achievable if and only if .
Layer 5: the transformation reading
Define by . The kernel is the null space, so and is not injective. The range is the column space, so and is not surjective. Rank plus nullity for transformations reproduces the matrix identity exactly.
Layer 4 is unreachable, and the chart says so
There is no meaningful question here about eigenvalues, the determinant, similarity or diagonalisation. Every theorem in layer four carries the hypothesis that the matrix is square, and is . Attempting or solving is not a hard computation — it is a type error. Recognising that boundary is exactly what the dependency chart is for.
The instrument set resolves a three-dimensional subspace of the six-dimensional state and is blind to a three-dimensional complement, so three further independent instruments would be required for full observability. One of the four existing instruments is redundant — its reading is forced by two others — which both provides a live consistency check on the hardware and identifies a sensor that could be removed without loss. All of this follows from a single row reduction and three layers of the chart; no spectral computation was performed or available.
