Engineering/Mathematics/Representations
Matrix Representations of Linear Transformations
A matrix representation converts a linear transformation between abstract vector spaces into an ordinary matrix, once a basis is fixed at each end. Every linear transformation then acts by a single matrix-vector product , which is what makes numerical linear algebra applicable to polynomials, matrices, functions and states.
- Advanced level
- Stream: representations
- Reading time 15 min
- Ref KVS-ENG-MATH-0109
- Taxonomy
- Engineering / Mathematics
- Prerequisite
- Bases, dimension and vector representation
- Produces
- An matrix
- Key theorem
- Depends on
- Both bases, and on the order of their vectors
- Payoff
- Every linear transformation becomes a matrix-vector product
Overview
A linear transformation can act on objects that are not columns of numbers: polynomials, matrices, signals, stress states, solutions of differential equations. None of that is directly computable. A matrix representation removes the obstruction by fixing an ordered basis for the domain and an ordered basis for the codomain, and recording what does to each basis vector of in the coordinate system supplied by . The result is a plain rectangular array of scalars that carries the entire content of the transformation.
The construction is deliberately economical. A linear transformation is determined by its values on a basis, because every input is a unique linear combination of basis vectors and linearity distributes across that combination. So only evaluations are needed, where is the dimension of the domain. Each evaluation lands in , and the vector representation turns it into a column of scalars. Stacking those columns side by side produces an matrix, and nothing further is required.
What justifies the word representation is a theorem rather than the definition. The Fundamental Theorem of Matrix Representation states that coordinatising the output of gives exactly the same answer as multiplying the matrix by the coordinatised input. The transformation and the matrix are therefore interchangeable, provided the bookkeeping of coordinates is respected at both ends. Read in the other direction, the theorem says that the action of any linear transformation on any finite-dimensional space can be executed as a matrix-vector product — the single most heavily optimised operation in scientific computing.
The price is that the matrix is not intrinsic. Change either basis and the entries change, often beyond recognition, while the transformation itself is untouched. That freedom is not a defect: it is the lever. Choosing bases well can turn a dense, opaque array into a diagonal one, and the search for the best possible representation of a given transformation is the organising question behind similarity, diagonalisation and canonical forms.
Definition
Matrix Representation of a Linear Transformation
MRLet be a linear transformation, let be an ordered basis of and let be an ordered basis of with elements. The matrix representation of relative to and is the matrix whose -th column is the coordinate vector, relative to , of the image of the -th basis vector of :
- evaluate in ;
- express that result as a linear combination of the vectors of ;
- write the resulting scalars as a column of length .
The definition is sensitive to the ordering of both bases. Permuting permutes the columns; permuting permutes the rows. A basis used for representation is therefore always an ordered list, not merely a set.
Vector Representation
VRFor a vector space with ordered basis , every has a unique expansion . The vector representation sends to the column of its coordinates . It is an invertible linear transformation, so no information is lost and reassembles the original object from its coordinates.
Linear Transformation
LTA function between vector spaces satisfying and for all vectors and all scalars . These two conditions are exactly what allow a transformation to be reconstructed from its values on a basis.
Concepts
Why the columns are images of basis vectors
Linearity means a transformation is determined by finitely many evaluations. Write an arbitrary input as ; then , so the images of the basis vectors are the only data that matter. The matrix-vector product is built to combine columns with exactly those weights: is the linear combination of the columns of using the entries of as scalars. Placing in column therefore makes the two operations agree by construction, not by coincidence.
The fundamental theorem and its commuting square
The central result is for every . Picture a square whose top edge is from to , whose left edge is from to , whose right edge is from to and whose bottom edge is multiplication by . The theorem says the two routes around the square agree: transform then coordinatise, or coordinatise then multiply. Rearranged as , it becomes an executable recipe — encode, multiply, decode.
Existence, uniqueness and the size of the matrix
Given the two ordered bases, the representation exists and is unique. Existence follows because is defined on all of ; uniqueness follows because coordinates relative to a basis are unique. The shape is forced: columns, one per domain basis vector, and rows, one per codomain basis vector. Note the asymmetry of the subscript convention — the domain basis indexes columns and the codomain basis indexes rows, matching the convention that consumes a column of length and returns a column of length .
One transformation, many matrices
Nothing distinguishes one admissible pair of bases from another, so a single transformation possesses an infinite family of matrix representations. They can look completely different: the same transformation may be represented by a dense array of large integers relative to an awkwardly chosen pair of bases and by a diagonal matrix relative to a well chosen pair. Quantities computed from the matrix therefore divide into two classes. Rank, nullity, invertibility and — when domain and codomain share a basis — eigenvalues and determinant are intrinsic to . Individual entries, sparsity, symmetry and condition number are artefacts of the bases.
The isomorphism behind the construction
Fixing and defines a map from the vector space of all linear transformations from to into the vector space of matrices. That map is itself linear and invertible: from any matrix one can build a transformation with that representation, and two transformations with identical representations are equal. Consequently , a fact that is awkward to prove directly but immediate once representations are available.
Why this makes abstract spaces computable
Practical engineering objects rarely arrive as columns. A displacement field discretised by shape functions, a polynomial calibration curve, a stress tensor and a truncated Fourier series all live in abstract finite-dimensional spaces. Matrix representation is the bridge that lets a solver written for operate on any of them: choose a basis, coordinatise, compute with the matrix, decode. Every finite element stiffness matrix, every state-space model and every discrete transform is a matrix representation of an underlying linear operator with respect to a chosen basis.
Procedure: building
Equations
Definition of the matrix representation
EQ-MR-01Columns are the coordinate vectors, relative to , of the images of the ordered basis .
Fundamental Theorem of Matrix Representation
EQ-MR-02Coordinatising the image equals multiplying the coordinatised input by the representation. This is the statement that licenses the term representation.
Executable form of the theorem
EQ-MR-03Encode the input, apply one matrix-vector product, decode the output. Absent the coordinate bookkeeping, a linear transformation simply is a matrix.
Entry formula
EQ-MR-04Entry is the coefficient of the -th codomain basis vector in the expansion of the image of the -th domain basis vector.
Dimensions of the representation
EQ-MR-05The domain basis indexes columns, the codomain basis indexes rows.
Dimension of the space of linear transformations
EQ-MR-06Because representation relative to fixed bases is an isomorphism from onto .
Differentiation on cubic polynomials
EQ-MR-07The derivative operator from to relative to the monomial bases and .
Variable Definitions
| Symbol | Name | Meaning | Domain / type |
|---|---|---|---|
| Linear transformation | The map being represented, from domain to codomain | element of L(U,V) | |
| Domain | Finite-dimensional vector space on which acts | vector space of dimension n | |
| Codomain | Vector space containing the images of | vector space of dimension m | |
| Domain basis | Ordered basis of ; indexes the columns | ordered basis of U | |
| Codomain basis | Ordered basis of ; indexes the rows | ordered basis of V | |
| Vector representation | Coordinatisation map from onto relative to | invertible linear transformation | |
| Matrix representation | The array encoding relative to and | m x n matrix | |
| Domain dimension | Number of vectors in , equal to the column count | positive integer | |
| Codomain dimension | Number of vectors in , equal to the row count | positive integer |
Worked Numerical Example
Problem statement
A polynomial model of beam curvature is stored as a cubic in . The design process needs its derivative, an element of . Build the matrix representation of the differentiation operator, use it to differentiate a specific cubic by matrix-vector product, then rebuild the representation on a second domain basis to see how the entries move while the transformation does not.
State the transformation and the bases
Let be differentiation, . It is linear because differentiation respects sums and scalar multiples. Take the monomial bases for the domain and for the codomain, so the representation will be .
Evaluate on the domain basis
Four evaluations exhaust the transformation: , , and .
Coordinatise each image relative to
Because is the monomial basis, each expansion is read off directly: , , and .
Differentiate by matrix-vector product
Take . Its coordinate vector is . Multiplying gives the coordinates of the derivative relative to .
Decode and check against direct differentiation
Applying to returns . Differentiating by hand gives the same polynomial, confirming the fundamental theorem on this input.
Replace the domain basis
Now use , still with as before. The images are , , and , which coordinatise to the columns below.
Confirm the same output from the new representation
Expanding the same in gives , so . The product reproduces the derivative exactly as before, even though the matrix and the coordinate vector both changed.
Read the structure from either matrix
Both matrices have rank and nullity , matching the transformation: differentiation on is surjective onto and annihilates exactly the constants. Those numbers are properties of ; the individual entries are properties of the bases.
Differentiation on cubics is faithfully executed by a matrix acting on coordinate vectors. Two different domain bases produce two different matrices that deliver identical results on every input, which is precisely the freedom exploited later when a basis is chosen to make the representation diagonal or banded.
Applications & Industry Use
State-space models as representations
A linear time-invariant plant is an operator on an abstract state space. The familiar , , and matrices are matrix representations of that operator relative to a chosen basis of state variables. Rewriting the state vector in modal or balanced coordinates is nothing more than changing the basis, which is why controllability and pole locations survive the change while the numerical entries do not.
Element stiffness matrices
The element stiffness matrix of a finite element is the representation of a linear operator mapping nodal displacement fields to nodal force fields, taken relative to the basis of shape functions. Choosing hierarchical or orthogonal shape functions instead of nodal ones changes the entries and the sparsity pattern of the assembled system without altering the underlying mechanics.
Transforms as basis changes
The discrete Fourier transform matrix, a wavelet analysis matrix and a discrete cosine transform matrix are all matrix representations of the identity or of a filtering operator between two bases of the signal space. Fast algorithms exist precisely because the representation relative to a well chosen basis is highly structured rather than dense.
Twists, wrenches and the adjoint
Rigid-body velocities form a six-dimensional vector space with no natural coordinates. Fixing a basis of screw axes turns the operator that transports a twist between frames into a adjoint matrix. Different conventions for ordering the linear and angular components are different basis orderings, which is why cross-library integration errors usually appear as permuted blocks.
Scene transforms on non-standard frames
A modelling transform is a linear map between object space and world space. Storing it as a matrix presupposes bases at both ends; a scene graph is effectively a chain of matrix representations composed along a path. Representing a transform in a tangent frame rather than in world axes is the standard trick for making normal mapping and skinning numerically well behaved.
Reaction networks in species coordinates
The map from reaction extents to species amounts is linear on an abstract space of chemical states. Choosing an ordered species list is choosing a basis, and the stoichiometric matrix is the resulting representation. Reordering species reorders rows, and reformulating in terms of independent reactions is a domain basis change that reveals the true rank of the network.
Design Considerations
Choose bases so that coordinatisation is trivial
Most of the labour in building a representation is not evaluating but expressing each image in the codomain basis. A basis such as the monomials for polynomials or the single-entry matrices for makes that step a matter of reading coefficients. A deliberately awkward basis turns each column into a separate linear solve, so prefer a convenient basis unless a specific structural payoff justifies the cost.
Record the bases with the matrix
A matrix representation is meaningless without the pair of ordered bases that produced it. Store the bases alongside the array in any serialised model, and treat the pair as a single artefact. Interface defects between subsystems overwhelmingly stem from two teams using the same matrix with different implicit orderings of the same basis vectors.
Decide whether entries or structure is the deliverable
If the deliverable is a numerical answer for one input, computing directly can be cheaper than building a representation at all. Representations pay for themselves when will be applied many times, when it must be composed with other transformations, or when structural quantities such as rank, kernel or eigenvalues are wanted.
Beware of badly conditioned bases
The transformation may be perfectly benign while the representation is numerically hostile. The monomial basis on a long interval is the classic example: coordinatisation relative to it is equivalent to solving with a Vandermonde matrix whose condition number grows exponentially with degree. An orthonormal basis keeps an isometry and prevents the coordinatisation step from dominating the error budget.
Exploit sparsity that a basis creates
Local bases — finite element hat functions, compactly supported wavelets, single-entry matrices — give representations in which most entries vanish. The saving is not cosmetic: sparse storage changes the memory and operation counts of every downstream product. When a representation must be assembled repeatedly, basis locality is usually the highest-leverage design choice available.
Keep domain and codomain bases distinct in notation
When it is tempting to use one basis for both roles, and for eigenvalue work that is mandatory. But when the two are genuinely different spaces, using symmetric-looking notation invites transposition errors. Retaining explicit subscripts throughout makes composition and inversion formulas self-checking, because adjacent subscripts must match.
Standards & Codes
Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.
| Reference | Title | Relevance to this topic |
|---|---|---|
ISO 80000-2 | Quantities and units — Part 2: Mathematics | Fixes the typographic conventions used here: italic for the transformation symbol and for scalar indices, upright for operator names such as , and bracketed arrays for matrices. |
BLAS Level 2 (GEMV) | Basic Linear Algebra Subprograms, matrix-vector operations | The fundamental theorem reduces the action of any linear transformation to one general matrix-vector product, which is exactly the operation standardised as xGEMV and tuned on every serious computing platform. |
IEEE 754-2019 | IEEE Standard for Floating-Point Arithmetic | Governs the rounding of the inner products that form each entry of a representation, and therefore the accuracy with which a coordinatisation step reproduces an image vector. |
ISO/IEC 40314 | Mathematical Markup Language (MathML) Version 3.0 | Encodes the sub-and-superscripted representation symbols and the matrix arrays on this page in machine-readable form for search and assistive technology. |
IEC 61131-3 | Programmable controllers — Part 3: Programming languages | Industrial control code implements state-space representations as fixed arrays; the standard's structured data types are where the ordered basis convention is either documented or silently lost. |
Material Selection
For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.
| Representation | Select when | Trade-off |
|---|---|---|
| Exact rational entries | Representations built symbolically from polynomial or matrix bases, where the intent is a structural result such as rank, kernel or an exact inverse. | Every entry is correct and pivot decisions are unambiguous, but coordinatisation against an awkward basis can produce large numerators and denominators. |
| IEEE 754 binary64 | Engineering models where the transformation itself comes from measured or discretised data, including all mainstream finite element and control workflows. | Fast and universally supported, but entries inherit the conditioning of the coordinatisation step, so a poor basis silently degrades every downstream result. |
| Complex scalars | Modal analysis, frequency-domain models and any setting where eigenvalues of the representation may be complex. | Doubles storage and roughly quadruples multiply cost, but avoids the artificial real block forms needed to keep oscillatory modes in real arithmetic. |
| Sparse storage with a local basis | Representations assembled from compactly supported shape functions or wavelet bases, where most basis pairs do not interact. | Memory and product cost scale with the number of non-zeros rather than , at the price of indexing overhead and irregular memory access. |
| Fixed-point entries | Embedded controllers executing a state update on hardware without a floating-point unit. | Deterministic timing and small code size, but the basis must be scaled in advance so that all entries and intermediate products stay inside the chosen range. |
| Orthonormal basis with unitary coordinatisation | Numerically sensitive work where the coordinate map should neither amplify nor attenuate magnitudes. | Coordinatisation becomes an inner product rather than a linear solve and conditioning is preserved, but the basis may lose the sparsity or physical interpretability of a nodal one. |
Manufacturing Notes
Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.
Cost of assembling a representation
Building requires evaluations of and coordinatisations. When the codomain basis is convenient the second step is free; otherwise each column is the solution of an system with the same coefficient matrix, so factorise the basis matrix once and reuse it for all right-hand sides rather than solving from scratch times.
Coordinatisation as a linear solve
For a codomain that is already with a non-standard basis , assemble the basis vectors as the columns of a matrix . Then is the solution of , and the whole representation is . Compute it with a factorisation and triangular solves; forming explicitly is slower and less accurate.
Verification by independent evaluation
The cheapest effective check is the fundamental theorem itself. Choose an input with no special structure, apply directly, and compare against the decode of the matrix-vector product. Because column influences the result only through coordinate of the input, testing with each basis vector in turn isolates a discrepancy to one column immediately.
Library behaviour
Symbolic systems support the construction directly: SymPy's polynomial and matrix classes make evaluation and coefficient extraction straightforward, and its Matrix objects accept the assembled columns. Numerical stacks such as NumPy and SciPy have no notion of an abstract basis, so the coordinatisation convention lives in user code — a strong argument for wrapping the basis pair and the array in a single object rather than passing bare arrays.
Numerical stability of the encode-decode cycle
Errors enter at three points: evaluating , solving for coordinates, and the final matrix-vector product. The middle step dominates when the basis is ill conditioned, since the relative error in the coordinates is bounded by the condition number of the basis matrix times the input error. Orthonormalising the basis, for example by a Gram-Schmidt or QR step, removes that amplification at a one-off cost.
Failure Modes & Common Mistakes
| Failure mode / mistake | Impact | Root cause | Prevention & detection |
|---|---|---|---|
| Swapping the roles of and | high | Coordinatising the images against the domain basis, or evaluating on the codomain basis, producing a transposed or nonsensical array. | Fix the mnemonic that the domain basis counts columns and the codomain basis counts rows, and check the shape is before proceeding. |
| Treating a basis as an unordered set | high | Two collaborators list the same basis vectors in different orders, so their matrices differ by row and column permutations. | Store bases as ordered sequences and version them with the matrix; never regenerate a basis from an unordered container such as a hash set. |
| Applying the matrix to the object instead of its coordinates | high | Forgetting the step and attempting to multiply the matrix by a polynomial, a matrix or a physical field. | Route every application through the executable form so the encode and decode steps cannot be skipped. |
| Comparing matrices built on different bases | medium | Concluding that two transformations differ because their arrays differ, when only the bases differ. | Compare intrinsic quantities such as rank and nullity, or first convert one representation to the other's bases with change-of-basis matrices. |
| Assuming the transformation is linear | high | Building a representation for a map with an offset or a product term, which then agrees with the true map only on the basis vectors. | Verify additivity and homogeneity explicitly on a non-basis input before starting; affine maps require an augmented construction, not a plain representation. |
| Using an ill-conditioned basis at high degree | medium | Monomial or equally spaced nodal bases at large dimension, where the coordinatisation solve is effectively a Vandermonde system. | Switch to an orthogonal family or a locally supported basis, and monitor the condition number of the basis matrix as dimension grows. |
| Mismatched dimensions in composition | medium | Composing representations whose intermediate bases are not the same, so the product is formed from incompatible coordinate systems. | Write the subscripts explicitly and require adjacent inner subscripts to match, exactly as with matrix dimensions. |
| Silent truncation of the codomain | low | Choosing a codomain basis that does not actually span the space containing every image, so some coordinatisations fail or are approximated. | Confirm that is a basis of a space genuinely containing the range of , not merely of a convenient subspace. |
FAQs
Why does the matrix depend on two bases rather than one?
Because the transformation has two ends. The input must be described in coordinates before the matrix can consume it, and the output arrives as coordinates that need interpreting. Those are separate acts of description, so they require separate bases — unless the domain and codomain are the same space and you deliberately use one basis for both, which is the setting for eigenvalues and similarity.
Is the matrix representation unique?
Relative to a fixed ordered pair of bases, yes, absolutely: coordinates relative to a basis are unique, so each column is determined. Across different choices of bases it is not unique at all, and a single transformation has infinitely many representations that may look nothing alike.
What happens if the domain and codomain have different dimensions?
Nothing unusual. The representation is simply rectangular, with rows and columns. Differentiation from cubics to quadratics gives a matrix; evaluation of a quadratic at two points gives a matrix. Only when the dimensions agree can the representation be square, which is a prerequisite for invertibility or eigenvalues.
Can I recover the transformation from the matrix?
Yes, provided you also know both bases. Apply to the matrix-vector product of the input's coordinates. Without the bases the array is uninterpretable, which is why the bases should be regarded as part of the data rather than as context that can be reconstructed later.
Does this only work for finite-dimensional spaces?
The construction as stated requires finite bases at both ends, so it is a finite-dimensional tool. Infinite-dimensional operators can be represented by infinite arrays relative to a Schauder or orthonormal basis, but convergence questions then arise that matrix algebra alone does not settle. In engineering practice the infinite-dimensional problem is discretised first, which is exactly the act of choosing a finite basis.
Why is this called the fundamental theorem rather than just a definition?
The definition only says how to build an array from evaluations. The theorem says that array reproduces the transformation on every input, not just on the basis vectors. That step needs linearity and the uniqueness of coordinates, and it is what makes matrix algebra a faithful calculus for linear transformations.
How do I choose good bases in practice?
Weigh three factors: how cheaply images can be coordinatised, how sparse or structured the resulting matrix is, and how well conditioned the coordinate map is. Nodal and monomial bases win on the first, local and eigen bases on the second, orthonormal bases on the third. Real workflows often carry two bases and a change-of-basis matrix between them for exactly this reason.
References
- Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section MR, Subsection MR. Licensed under the GNU Free Documentation License v1.2.
- ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
- Golub, G. H. and Van Loan, C. F. Matrix Computations, 4th edition. Johns Hopkins University Press, 2013.
- Anderson, E. et al. LAPACK Users' Guide, 3rd edition. Society for Industrial and Applied Mathematics, 1999.
- IEEE 754-2019, IEEE Standard for Floating-Point Arithmetic. Institute of Electrical and Electronics Engineers.
AI Suggested Questions
- Build the matrix representation of the operator that maps a quadratic to its antiderivative with zero constant term, relative to the monomial bases, and explain the shape of the result.
- Show me two bases for which the representation of the same transformation is dense in one case and diagonal in the other.
- How does the condition number of the basis matrix affect the accuracy of a coordinatisation step, with a numerical example at degree 10?
- Explain how a finite element stiffness matrix is a matrix representation, and identify precisely which basis is being used.
- Why does the space of linear transformations from an -dimensional space to an -dimensional space have dimension ?
- Given a representation and both bases, reconstruct a closed-form algebraic expression for the transformation itself.
Related Calculators
Enter a linear transformation and two ordered bases to assemble column by column, with each coordinatisation shown.
Vector Coordinatisation CalculatorCompute for polynomials, matrices or column vectors relative to any supplied basis, and invert the map.
Basis Condition Number ToolAssemble a basis matrix and report its condition number to gauge how much error a coordinatisation step will amplify.
