Engineering/Mathematics/Matrices
The Four Fundamental Subsets of a Matrix
Every matrix carries four subspaces: two in describing its unknowns, two in describing its equations. One reduction of the augmented array produces two small matrices, and , from which all four follow with no further elimination.
- Advanced level
- Stream: subspaces
- Reading time 16 min
- Ref KVS-ENG-MATH-0054
- Taxonomy
- Engineering / Mathematics
- Prerequisite
- Extended echelon form, null space and row space bases
- The four sets
- , , ,
- Dimensions
- , , ,
- Computed from
- and alone, both already reduced
- Cost
- One reduction of an array
Overview
Four subspaces attach to every matrix. The null space collects the vectors annihilated by ; the row space collects the linear combinations of its rows. Both live in and describe the unknowns. The column space collects the achievable right-hand sides; the left null space collects the combinations of rows that vanish. Both live in and describe the equations.
Computed separately, each requires its own reduction and its own bookkeeping: one for the null space, one for the row space, a transpose and another reduction for the column space, and a further one for the left null space. Four reductions of three different matrices, with four opportunities for the pivot patterns to disagree.
The organising result replaces all of that with one computation. Reduce to extended echelon form and partition the result at row . Then — the non-zero rows of the reduced — determines both subspaces in , and — the trailing rows of the appended block — determines both subspaces in . The theorem is a striking symmetry: gives a null space and a row space, and so does , but the roles are exchanged.
Both and arrive already in reduced row-echelon form with no zero rows, which is exactly the state in which a null space basis and a row space basis are cheapest to extract. In practice the theorem is applied and then followed by two null space extractions and two row space extractions on small, clean matrices. The dimensions come out automatically: , , and , adding to within and to within .
Definition
The Four Subsets
FSLet be an matrix whose reduced row-echelon form has non-zero rows, and let be its extended echelon form. Let be the submatrix of formed from the first rows and first columns, and the submatrix formed from the last rows and last columns. Then
- ;
- ;
- ;
- .
The first two are near-immediate; the second two are the substantive content. Note the exchange of roles: supplies a null space and a row space in , and supplies a null space and a row space in .
Full Rank
FRA matrix has full rank when , that is, when its reduced row-echelon form has no zero rows. Then has no rows at all, there are no consistency conditions, and with . For a square matrix, full rank coincides with nonsingularity.
When is empty, should be read as the solution set of a homogeneous system with no equations, which every vector satisfies — hence all of .
Basis of a Null Space
BNSFor a matrix in reduced row-echelon form with pivot columns and free columns , setting one free variable to and the remainder to , then solving for the pivot variables, produces one vector per free column. The resulting set is linearly independent and spans the null space.
Basis of a Row Space
BRSFor a matrix in reduced row-echelon form, the non-zero rows written as column vectors form a linearly independent set spanning the row space. When the matrix has no zero rows — as both and are guaranteed to be — every row is used.
Concepts
Why captures both subspaces in
The reduced form is row-equivalent to , so it has the same null space and the same row space. Removing the zero rows of to obtain changes neither: a zero row corresponds to the equation , whose removal cannot change a solution set, and it contributes the zero vector to a span, whose removal cannot change the span. Hence and . These two conclusions are close to trivial, but they set the pattern that the harder half of the theorem mirrors.
Why the column space is the null space of
Take , so for some . The extended form guarantees , and the last rows of are zero, so the last entries of vanish. Those entries are exactly , giving . Conversely, if , then the system with coefficient matrix and constants is consistent — is reduced with no zero rows, so no contradiction can arise — and any solution satisfies , hence . The two inclusions give equality.
Why the left null space is the row space of
The rows of are the scalars expressing the rows of as combinations of the rows of , an identity recorded in . The rows of are those of aligned with the zero rows of , so each is a set of scalars producing a vanishing combination of the rows of — precisely the defining property of a left null space vector. That gives . Equality follows by a dimension count, or directly: any can be expressed through the nonsingular , and the independence of the rows of forces the leading part of that expression to vanish, leaving in the span of the rows of .
The dimension arithmetic
is in reduced form with no zero rows, so it has pivot columns and free columns, giving and . is in reduced form with no zero rows, so it has pivot columns and free columns, giving and . The two pairs sum to and to respectively, and the equality falls out as a by-product rather than requiring a separate argument.
The full-rank case
When the reduced form of has no zero rows, so has no rows. The column space is then the null space of a homogeneous system with no equations, which every vector satisfies, so . The same conclusion is reached without the empty-matrix argument: reducing with produces no zero rows and therefore no pivot in the final column, so the system is consistent for every . Correspondingly — no combination of rows cancels.
Three descriptions of one column space
This theorem supplies the third route to a column space basis, alongside taking pivot columns of the original matrix and transposing before reducing. The three produce different-looking bases with different virtues. Pivot columns are actual columns of and retain physical meaning. The transposed route places leading ones near the top of each vector. The route through places them near the bottom. All three have elements and span the same subspace; which to ship is a presentation decision.
Extracting all four subsets from one reduction
Equations
The partition
EQ-FS-01is , is , and both are in reduced row-echelon form with no zero rows.
The two subspaces in
EQ-FS-02Both follow from row equivalence together with the harmlessness of deleting zero rows.
The two subspaces in
EQ-FS-03Note the exchange of roles relative to : the column space appears as a null space, and the left null space as a row space.
Dimensions of the four subsets
EQ-FS-04All four follow from the pivot and free column counts of and , with no separate computation.
The two dimension identities
EQ-FS-05One identity for each ambient space. The first is the rank-nullity relation; the second is its counterpart on the equation side.
Consistency stated through
EQ-FS-06A restatement of using the equivalence between membership of the column space and solvability.
Full rank
EQ-FS-07has no rows, so there are no conditions to satisfy and no vanishing combination of rows. For a square matrix this is nonsingularity.
Variable Definitions
| Symbol | Name | Meaning | Domain / type |
|---|---|---|---|
| Matrix | The matrix whose four subsets are sought | m x n matrix over C | |
| Rank | Number of non-zero rows of the reduced form of ; sets every dimension on this page | 0 to min(m,n) | |
| Top-left block | The non-zero rows of the reduced ; governs the two subspaces in | r x n matrix | |
| Bottom-right block | The trailing rows of the appended block; governs the two subspaces in | (m-r) x m matrix | |
| Null space | Vectors annihilated by ; the ambiguity in the unknowns | subspace of C^n, dimension n-r | |
| Row space | Linear combinations of the rows; the estimable directions | subspace of C^n, dimension r | |
| Column space | Achievable right-hand sides; the reachable outputs | subspace of C^m, dimension r | |
| Left null space | Vanishing combinations of rows; the redundancy in the equations | subspace of C^m, dimension m-r | |
| Transforming matrix | The reduced appended block, nonsingular with ; is its trailing rows | m x m nonsingular matrix |
Worked Numerical Example
Problem statement
A four-node supply network is driven by three settings, giving a 4 by 3 map from settings to nodal demands. Determine all four fundamental subsets from a single reduction, and interpret each one in operational terms.
Reduce the augmented array
With and , adjoin and reduce the whole array, continuing until the appended block is also in reduced form.
Read off and
The reduced has two non-zero rows, so . Take the top rows of the first three columns as , and the bottom rows of the last four columns as . Everything below follows from these two matrices alone.
Null space from
has pivot columns and and one free column, . Setting gives and . Checking against the original matrix: on the first row, and similarly on the other three.
Row space from
has no zero rows, so both rows transpose directly into a basis. Each is a combination of the original rows: the first equals and the second equals .
Column space from
has pivot columns and and free columns and . Setting each free variable to in turn gives two basis vectors, with the identity pattern sitting in the trailing coordinates.
Left null space from
has no zero rows, so both rows transpose into a basis. The first asserts , which checks: . The second asserts , which also checks.
Verify the dimension identities
In : . In : . Both identities hold, confirming the partition boundary and all four extractions.
Cross-check the column space against the columns
Every column of must satisfy . For the first column, and ; for the second, and ; for the third, and . All three pass.
The network reaches a two-dimensional family of nodal demands out of four possible dimensions, using only two of its three settings independently: the null space vector is the setting combination that produces no demand at all, and the two rows of are the conservation relations that every achievable demand must satisfy. Rank appears simultaneously as the number of independent settings, the number of independent equations and the dimension of the reachable demand space — one reduction, four answers.
Applications & Industry Use
Static and kinematic indeterminacy together
For an equilibrium matrix relating member forces to nodal loads, the null space holds the self-stress states (static indeterminacy), the column space holds the loads that can be equilibrated, and the left null space holds the mechanisms — nodal motions that no member restrains. Extracting all of them from one reduction gives a complete structural classification in a single pass.
Reachability and unobservable directions
For a controllability matrix, the column space is the reachable subspace and the left null space identifies the state directions no input can influence. For an observability matrix the roles transfer to the row space and null space. Presenting all four simultaneously makes the duality between the two properties explicit rather than a separate theorem.
Reaction invariants and conservation laws
With a stoichiometric matrix, the column space is the set of composition changes any reaction sequence can produce, and the left null space is the set of conserved quantities — elemental balances and moieties preserved by every reaction. The null space identifies reaction combinations that produce no net change, useful for detecting futile cycles.
Datum defect and estimable functions
In a levelling or geodetic network, the null space of the design matrix is the datum defect — the coordinate shifts that no observation can detect. The row space is the set of estimable functions, and the left null space gives the loop closures used to assess the observations. Isolating all four clarifies whether an ill-posed adjustment is a datum problem or a data problem.
Loops, cutsets and Kirchhoff structure
For a node-arc incidence matrix, the null space is the cycle space, the row space is the cutset space, the column space is the set of realisable nodal injections, and the left null space encodes global conservation. The four subsets reproduce the classical Kirchhoff decomposition of a network directly from the matrix.
Identifiability of a linear model
For a design matrix, the null space is the set of parameter changes invisible to the data, so it is exactly the non-identifiable directions. The row space is what the data can determine, the column space is what the model can fit, and the left null space is the residual structure no parameter setting can explain. Reporting all four converts a vague complaint about collinearity into a precise statement.
Design Considerations
Keep the ambient spaces straight
The null space and row space are subspaces of ; the column space and left null space are subspaces of . For a rectangular matrix a vector from one pair cannot even be compared with one from the other. Recording the sizes alongside the bases prevents an entire class of errors, and a shape check is the cheapest available verification.
The theorem locates the subspaces; it does not compute the bases
The conclusion expresses each subset as a null space or row space of or . Turning that into an explicit basis still requires the standard extractions. The saving is that those extractions run on small matrices already in reduced form, so no further elimination is needed — not that the bases appear for free.
Choose a column space description deliberately
This route puts the identity pattern in the trailing coordinates of each basis vector; the transpose route puts it in the leading coordinates; the pivot-column route uses actual columns of . All three are correct and all three have elements. Pick according to whether physical interpretability, compact presentation or trailing-coordinate structure serves the deliverable.
Interpret the empty correctly
When has full row rank, has no rows. This is not an error state and it does not mean the column space is trivial — it means the opposite. A homogeneous system with no equations is satisfied by everything, so . Code that iterates over the rows of must handle the zero-row case explicitly rather than failing or returning an empty span.
Cost is justified only when several subsets are wanted
The extended reduction roughly doubles the arithmetic of an ordinary reduction for a square matrix. If only a null space is needed, reduce and extract; if only a rank is needed, reduce and count. The construction earns its cost when two or more of the four subsets are required, or when the transforming matrix is itself of interest.
For measured data, use the SVD analogue
Every dimension on this page depends on an exact value of , which floating-point data cannot supply. The numerically robust counterpart is the singular value decomposition: leading right singular vectors span the row space, trailing right singular vectors span the null space, leading left singular vectors span the column space and trailing left singular vectors span the left null space — all orthonormal, with the singular-value gap justifying the split.
Standards & Codes
Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.
| Reference | Title | Relevance to this topic |
|---|---|---|
ISO 80000-2 | Quantities and units — Part 2: Mathematics | Fixes the calligraphic notation distinguishing the four subsets and the partitioned-matrix conventions used to present the block decomposition. |
LAPACK / BLAS reference | Linear Algebra PACKage reference implementation | Provides the production route to all four subspaces through xGESVD, whose left and right singular vector blocks partition into exactly these four orthonormal bases. |
IEEE 754-2019 | IEEE Standard for Floating-Point Arithmetic | Determines the rounding behaviour that makes the rank — and therefore all four dimensions on this page — a tolerance-dependent quantity for measured data. |
ISO 17123 | Optics and optical instruments — Field procedures for testing geodetic and surveying instruments | Underpins network adjustment practice, where the null space appears as datum defect and the left null space as the loop misclosures used to qualify observations. |
ISO/IEC 40314 | Mathematical Markup Language (MathML) Version 3.0 | Encodes the set-valued conclusions and partitioned arrays semantically so that the distinction between , , and survives into assistive technology. |
Material Selection
For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.
| Representation | Select when | Trade-off |
|---|---|---|
| Exact rational arithmetic | Structural classification of a small model where the four dimensions are the deliverable and must be certain. | Gives an unambiguous rank and integer bases that read as physical relations, but entries in the appended block can grow and memory use is unbounded. |
| Arbitrary-precision integers | Incidence, stoichiometric or equilibrium matrices where bases should be reported with integer entries. | Preserves interpretability of conservation relations and self-stress states; needs a normalisation convention for sign and scale of each basis vector. |
| IEEE 754 binary64 with SVD | Measured design matrices, identifiability analysis and any case where the rank is uncertain. | Yields four orthonormal bases with a quantified rank gap and excellent conditioning, at roughly an order of magnitude more arithmetic than elimination. |
| IEEE 754 binary64 with pivoted QR | Large dense matrices where an SVD is too costly but numerically sound bases are still needed. | Gives the column space and left null space cheaply and reliably; the null space and row space require a second factorisation of the transpose. |
| Finite field arithmetic | Coding theory, where becomes a generator matrix and a parity-check matrix for the same code. | Exact and fast with no coefficient growth, and the four subsets map directly onto code, dual code and their descriptions; restricted to the algebraic setting. |
| Sparse factorisation with implicit bases | Very large sparse network or finite-element models where explicit bases cannot be stored. | Keeps memory tractable by representing subspaces through operators rather than explicit vectors, at the cost of losing direct inspection of the basis entries. |
Manufacturing Notes
Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.
What the theorem costs and what it saves
One extended reduction of an array replaces up to four separate reductions of three different matrices. For a square input the single reduction costs roughly twice an ordinary one, so the break-even point is two subsets. Beyond the saving in arithmetic, deriving everything from one partition guarantees the four results are mutually consistent, which independent computations do not.
Doing it by hand
Reduce the full array, then write and out separately before extracting anything. Attempting to read null spaces and row spaces directly off the large array is where errors enter, because the pivot indices of are numbered within its own columns, not within the columns of . Two small clean matrices on a fresh line are worth the extra transcription.
Verification technique
Four independent checks cover the result. Confirm for each null space basis vector. Confirm each row space basis vector is a combination of the rows of . Confirm for every column of . Confirm for each left null space basis vector. Then check that the dimensions sum to and to .
Library behaviour
SymPy exposes nullspace(), rowspace() and columnspace() individually, with the left null space obtained as A.T.nullspace(); there is no combined routine. In SciPy the standard approach is a single scipy.linalg.svd with full_matrices=True, after which all four bases are slices of and at the index determined by the rank tolerance — the numerical counterpart of this theorem.
Numerical fragility of the partition
Every dimension here hinges on the single integer . An error of one moves a basis vector between the row space and the null space, or between the column space and the left null space, and both errors are silent. Any production use must report the evidence for the chosen rank — a singular-value gap, a pivot magnitude ratio — alongside the bases themselves.
Failure Modes & Common Mistakes
| Failure mode / mistake | Impact | Root cause | Prevention & detection |
|---|---|---|---|
| Mixing vectors from the two ambient spaces | high | Treating a null space vector and a column space vector as comparable, when one has entries and the other . | Label every basis with its ambient space and check lengths. For rectangular matrices the mismatch is caught by shape; for square ones it is not. |
| Reading as the top rows of | high | Confusing with when partitioning the appended block. | aligns with the zero rows of the reduced . Verify with for every column; fails this test. |
| Treating an empty as a trivial column space | high | Full row rank leaves with no rows, and code or reasoning that iterates over its rows concludes the column space is empty or trivial. | A homogeneous system with no equations is satisfied by every vector, so . Handle the zero-row case explicitly. |
| Numbering the pivot columns of within | medium | Using column indices from the full extended array when extracting the null space of , so the free-variable assignments are misplaced. | Write out as a standalone matrix and index its columns from before any extraction. |
| Assuming the row space and column space are equal | medium | Inferring set equality from the shared dimension . | They are subspaces of different spaces unless , and even then they are usually distinct. Equal dimension is a theorem; equal sets is a coincidence. |
| Skipping the standardising reduction | medium | Halting once the left block is reduced, so is not canonical and results cannot be reproduced or compared. | Continue the reduction into the appended block until the whole array satisfies the reduced row-echelon conditions. |
| Exact rank assumption on measured data | high | Partitioning at a rank determined by exact-zero pivot tests on floating-point entries. | Use an SVD with a documented tolerance and report the singular-value gap that justifies the partition boundary. |
| Omitting the zero vector from a trivial subspace | low | Reporting an empty basis as an empty set and then treating the subspace as containing nothing. | An empty basis spans , not the empty set. Every subspace contains the zero vector by definition. |
FAQs
Why are these four subsets the natural ones to single out?
Each answers a distinct structural question. The null space measures ambiguity in the unknowns; the row space measures what the equations can determine; the column space measures what right-hand sides are achievable; the left null space measures redundancy among the equations. Together they account for both ambient spaces completely, since their dimensions sum to and to .
Why does give a null space and a row space, and do the same thing again?
Because both are matrices in reduced row-echelon form with no zero rows, so both admit the same two standard extractions. The symmetry is genuine, but the meanings are exchanged: for the null space is and the row space is , whereas for the null space is the column space of and the row space is the left null space.
Do I have to use the extended echelon form?
No. Each subset can be computed separately with its own reduction — reduce for the null space and row space, transpose and reduce for the column space and left null space. The extended form is a consolidation: one reduction rather than several, and four mutually consistent results rather than four independent ones.
What happens when the matrix has full row rank?
has no rows. There are no consistency conditions, so every right-hand side is achievable and ; correspondingly no combination of rows cancels, so . For a square matrix this coincides with nonsingularity.
Why do the row space and column space always have the same dimension?
On this route it emerges from the pivot counts: has rows and therefore pivot columns, giving , while has pivot columns and hence free columns, giving . The equality is a consequence of the partition rather than a separate theorem.
Does this give me the best possible basis for each subset?
It gives a correct and linearly independent basis for each, with a convenient zero-one pattern. It is not orthonormal, not well-conditioned for floating-point work, and not made of original columns. For numerical work an SVD produces orthonormal bases for all four; for physical interpretation, the pivot columns of give a column space basis of real columns.
How do I check that my four bases are right?
Verify each one against the original matrix: null space vectors must satisfy ; row space vectors must be combinations of the rows; column space vectors must satisfy and be reachable as ; left null space vectors must satisfy . Then confirm the dimensions sum to and to .
References
- Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section FS, Subsection FS. Licensed under the GNU Free Documentation License v1.2.
- ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
- Strang, G. Linear Algebra and Its Applications, 4th edition. Cengage Learning, 2006.
- Golub, G. H. and Van Loan, C. F. Matrix Computations, 4th edition. Johns Hopkins University Press, 2013.
- Trefethen, L. N. and Bau, D. Numerical Linear Algebra. Society for Industrial and Applied Mathematics, 1997.
- Anderson, E. et al. LAPACK Users' Guide, 3rd edition. Society for Industrial and Applied Mathematics, 1999.
AI Suggested Questions
- Take a 5x4 matrix of rank 3 and compute all four fundamental subsets from one extended reduction, checking every dimension identity.
- Map the four subsets of a singular value decomposition onto the four subsets described here, and explain what changes when the rank is uncertain.
- For a node-arc incidence matrix of a small graph, identify the cycle space and cutset space among the four subsets and explain the correspondence.
- Why does the column space appear as a null space in this theorem rather than as a span, and what does that buy computationally?
- Show what happens to C and L when the matrix has full column rank rather than full row rank, and describe each of the four subsets in that case.
- Explain how the four subsets diagnose non-identifiability in a linear regression design matrix, with a concrete collinear example.
Related Calculators
Compute bases for , , and from one extended reduction, with dimension checks.
Extended Echelon Form CalculatorReduce and return , , , and with a verification of .
Matrix Rank & Nullity CalculatorReport rank, nullity and left-nullity with a selectable rank tolerance.
