Engineering/Mathematics/Eigenvalues
Polynomials and Matrices
A polynomial needs only addition, scalar multiplication and powers, and square matrices support all three. Substituting a matrix for the variable is therefore natural, provided the constant term is read as a multiple of the identity.
- Advanced level
- Stream: eigen
- Reading time 12 min
- Ref KVS-ENG-MATH-0079
- Taxonomy
- Engineering / Mathematics
- Applies to
- Square matrices only
- Constant term
- becomes , never a scalar
- Convention
- Key property
- Factored and expanded forms agree
- Eigen link
Overview
A polynomial is built from three ingredients: powers of the variable, multiplication by scalar coefficients, and addition. Division never appears. Square matrices support exactly the same three operations — they can be multiplied by scalars, added to one another, and raised to positive integer powers by repeated matrix multiplication — and, crucially, all of those operations preserve the size of a square matrix. Substituting a matrix for the variable of a polynomial is therefore a well-defined operation with a matrix result.
The only genuine subtlety is the constant term. In the scalar polynomial the leading term is a number; in the matrix version it must be a matrix of the right size, and the only sensible candidate is . This follows from the convention , which makes the constant term the degree-zero term like any other. Writing with a bare scalar is a common abbreviation, but it is an abbreviation, and taking it literally in code is one of the reliable ways to compute the wrong answer.
The second observation is that the factored form of a polynomial may be substituted just as freely as the expanded form, and gives the same matrix. That is not automatic: matrix multiplication does not commute in general, so expanding a product of matrix factors normally requires care about ordering. Here it is safe, because every factor is a polynomial in the single matrix , and any two polynomials in the same matrix commute with one another. The factored form is often both cheaper to evaluate and far more informative.
Matrix polynomials are not an idle generalisation. They are the machinery behind the proof that every square matrix has an eigenvalue, behind the Cayley-Hamilton theorem, behind Krylov subspace iterative solvers, behind pole-placement formulas in control, and behind closed-form rotation formulas in kinematics. Wherever a matrix appears inside an expression built from sums, scalar multiples and powers, a matrix polynomial is what is actually being evaluated.
Definition
Polynomial Evaluated at a Matrix
PMLet be a polynomial with coefficients in , and let be a square matrix of size . Then
- ,
The requirement that be square is essential twice over: a non-square matrix cannot be multiplied by itself, and the sum requires the identity to have the same shape as .
Matrix Power
MPFor a square matrix and a non-negative integer , the power is defined recursively by and for . The convention is what makes the index laws hold without exception, and it is the reason the constant term of a polynomial becomes a multiple of the identity.
This is not the entrywise power. Raising each entry of to the power gives a completely different matrix, and several programming languages make the entrywise operation the default for the ** operator.
Annihilating Polynomial
APA non-zero polynomial with , the zero matrix, is said to annihilate . A weaker and equally useful notion fixes a single vector: annihilates at when . Such polynomials always exist, and factoring them is the route by which eigenvalues can be extracted without ever computing a determinant.
Concepts
The constant term is a matrix
Every term of must be an matrix, because they are being added together. The terms , and so on are matrices automatically. The degree-zero term is , and since this is the matrix : the scalar on the diagonal and zeros everywhere else. It is not the scalar , and it is emphatically not the matrix with in every position. This distinction is invisible in the standard abbreviated notation and highly visible in a wrong answer.
Why the factored form may be substituted
Matrix multiplication is not commutative, so expanding appears to demand care. It does not, because commutes with itself and with every multiple of the identity. Consequently any two polynomials in the same matrix commute: , and both equal , the polynomial product evaluated at . Substituting into a factored polynomial therefore gives exactly the same matrix as substituting into its expanded form, and the factors may be applied in any order.
The factored form is usually the better one
Beyond convenience, the factored form carries structure. Each factor is a shifted matrix, and its null space is the eigenspace of for the eigenvalue . A product of such factors annihilates any vector lying in the eigenspace of any of the roots. That observation drives the constructive proof that every square matrix has an eigenvalue: build an annihilating polynomial for a chosen vector, factor it, and apply the factors one at a time until the result becomes the zero vector. The last factor applied identifies an eigenvalue.
Action on an eigenvector
If then by repeated application, and therefore for every polynomial . An eigenvector of is an eigenvector of every polynomial in , with the eigenvalue transformed by the same polynomial. This is the spectral mapping property in its most elementary form, and it explains why forces every eigenvalue of to be a root of .
Degree can always be reduced
The powers cannot be independent forever: the space of matrices has dimension , so at most of them can be linearly independent, and the Cayley-Hamilton theorem sharpens this dramatically by showing that is a combination of the lower powers. Every polynomial in therefore equals a polynomial of degree less than . In practice this means that evaluating a high-degree matrix polynomial should begin by reducing the polynomial modulo the characteristic polynomial, not by computing high powers.
Powers are where the numerical trouble lives
The entries of grow or decay roughly like , where is the spectral radius. For the powers overflow; for they underflow to zero; and for a non-normal matrix the intermediate powers can be enormous even when both and are modest, producing catastrophic cancellation when the terms are summed. Matrix polynomials are also destructive of sparsity: each multiplication by propagates non-zeros further from the diagonal, so of a sparse matrix is generally dense.
Evaluating a matrix polynomial
Equations
Polynomial evaluated at a matrix
EQ-PM-01Each term is an matrix. The degree-zero term is , not the scalar .
Matrix powers
EQ-PM-02The recursive definition of a matrix power and the index law it satisfies. The convention is what makes the index law hold for .
Factored form substituted
EQ-PM-03Over every polynomial factors into linear factors. The matrix factors commute, so the ordering is immaterial and the product equals the expanded form.
Polynomials in one matrix commute
EQ-PM-04The reason the factored substitution is legitimate. It fails for polynomials in two different matrices unless those matrices themselves commute.
Action on an eigenvector
EQ-PM-05An eigenvector of is an eigenvector of every polynomial in , with the eigenvalue mapped through the same polynomial.
Horner's scheme for a matrix argument
EQ-PM-06Evaluates a degree- polynomial in matrix multiplications without ever forming a high power explicitly, which limits both cost and overflow.
Annihilating polynomial at a vector
EQ-PM-07The relation that a linear dependence among produces. Factoring such a and applying the factors in turn extracts an eigenvalue of .
Variable Definitions
| Symbol | Name | Meaning | Domain / type |
|---|---|---|---|
| Matrix argument | The square matrix substituted for the polynomial variable | n x n matrix over C | |
| Size | Common row and column count of and of | positive integer | |
| Polynomial | The scalar polynomial whose variable is replaced by a matrix | polynomial with complex coefficients | |
| Degree | Degree of the polynomial, the highest power appearing | non-negative integer | |
| Coefficient | Scalar multiplier of the -th power term | complex scalar | |
| Root | A root of , appearing in the factor | complex scalar | |
| Identity matrix | The size- identity, equal to and carrier of the constant term | n x n matrix | |
| Matrix power | The -fold matrix product of with itself; not the entrywise power | n x n matrix | |
| Zero matrix | The matrix with every entry zero, the result when annihilates | n x n matrix |
Worked Numerical Example
Problem statement
A discrete-time state transition matrix in companion form governs a third-order digital filter. Evaluate the quadratic at that matrix, first from the expanded form and then from the factored form , and interpret the structure of the result.
State the matrix and the polynomial
The companion matrix shifts the state and forms the new entry from the feedback coefficients in its last row. The polynomial to be evaluated is , whose constant term will contribute .
Compute the required powers
Only , and are needed. Because the first two rows of are shifted unit vectors, the first two rows of are simply the second and third rows of ; only the last row requires arithmetic, as .
Assemble the expanded form
Combine the three matrices with the coefficients , and . The constant term enters as , contributing to each diagonal entry and nothing elsewhere — not to every entry.
Complete the entrywise arithmetic
Adding the three matrices entry by entry gives the result. For instance the entry is , and the entry is .
Form the factored version
The polynomial factors as , so . Each factor is with a constant subtracted from the diagonal only.
Multiply the factors and compare
The first row of the product is , matching the expanded result. The remaining rows agree likewise. Because polynomials in commute, reversing the two factors gives the same matrix.
Read the structure of the result
Every row of is a multiple of and every column is a multiple of , so has rank one. That column direction is not accidental: , so it is an eigenvector of for the eigenvalue — the one root of the cubic that does not remove.
Extend to the annihilating polynomial
Multiplying by the remaining factor gives the zero matrix, since has first row and the other rows are multiples of the first. Hence for , which is exactly the characteristic polynomial encoded in the last row of the companion matrix.
The quadratic evaluates to a rank-one matrix whose column space is the eigenspace of for the eigenvalue : the factors and have annihilated the other two eigendirections and left only the third. Applying the final factor annihilates that one too, so the cubic is an annihilating polynomial for . For the filter this is the statement that its state transition matrix satisfies its own difference equation — the feedback coefficients in the last row of are precisely the coefficients of the polynomial that kills it.
Applications & Industry Use
Pole placement and state transition
Ackermann's formula computes a state-feedback gain as a row vector times the desired characteristic polynomial evaluated at the plant matrix, . Separately, the discrete-time state transition over steps is , and Cayley-Hamilton lets that power be written as a polynomial of degree less than , which is how transition matrices are evaluated without repeated multiplication.
Closed-form rotation from a skew matrix
Rodrigues' formula expresses a rotation about a unit axis as , where is the skew-symmetric cross-product matrix of the axis. This is a quadratic polynomial in ; higher powers are unnecessary because collapses them back. Evaluating it is a matrix polynomial evaluation performed millions of times per second in motion planning code.
Krylov subspace iterative solvers
Conjugate gradients, GMRES and their relatives construct approximate solutions of the form for a polynomial of degree . The residual after steps is with , and the entire convergence theory of these methods is the question of how small such a polynomial can be made on the spectrum of .
Walk counting and graph polynomials
For an adjacency matrix , the entry counts walks of length exactly from node to node . A polynomial in therefore counts weighted combinations of walks, which is how communicability, Katz centrality and network resilience measures are defined and computed in transport and communication network engineering.
Filters as polynomials in a shift operator
A finite impulse response filter applied to a finite signal is a polynomial in the shift matrix, with the filter taps as coefficients. Cascading two filters corresponds to multiplying their polynomials, and because polynomials in one matrix commute, the order in which two filters are applied does not change the result — a fact filter designers rely on when reordering a processing chain.
Polynomial approximation of matrix functions
Transient heat conduction and structural response require the matrix exponential , which is approximated in practice by a truncated or Chebyshev polynomial in , or by a rational approximation whose numerator and denominator are both matrix polynomials. The evaluation strategy — Horner, Paterson-Stockmeyer, scaling and squaring — is chosen entirely on matrix-polynomial grounds.
Design Considerations
Write the constant term as in code as well as on paper
In array-based languages, adding a scalar to a matrix broadcasts: A + 2 adds to every entry, not to the diagonal. The correct expression is A + 2*np.eye(n). The bug produces a plausible matrix of the right shape and is easily missed in review, so it should be caught by an explicit unit test on a diagonal input.
Reduce the degree before evaluating
By Cayley-Hamilton, any polynomial in equals one of degree less than . For a high-degree polynomial, divide by the characteristic polynomial and evaluate only the remainder. This can turn an evaluation requiring hundreds of matrix multiplications into one requiring a handful, and it also avoids forming the high powers where overflow lives.
Never form high powers explicitly
Computing by nineteen multiplications is both slow and numerically poor. Horner's scheme evaluates a degree- polynomial in multiplications with no explicit high powers; the Paterson-Stockmeyer scheme reduces that to about . Where only itself is wanted, binary exponentiation needs about multiplications.
Expect sparsity to be destroyed
Each multiplication by a sparse spreads non-zeros further, so is generally dense even when is not. Matrix polynomials of sparse matrices should therefore be applied to vectors rather than formed as matrices: computing by repeated sparse matrix-vector products costs sparse products and never materialises a dense intermediate.
Prefer the factored form when the roots are known and real
Forming requires only diagonal subtractions plus multiplications, avoids computing coefficients that may be enormous, and exposes the eigenstructure. If the roots are complex and the matrix real, pair conjugate roots into real quadratic factors to keep the arithmetic real.
Watch conditioning for non-normal matrices
For a matrix that is far from normal, intermediate powers can be many orders of magnitude larger than either the matrix or the final answer, so summing the terms of an expanded polynomial cancels catastrophically. Where accuracy matters, evaluate through a Schur decomposition or a scaling-and-squaring scheme rather than by direct summation.
Standards & Codes
Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.
| Reference | Title | Relevance to this topic |
|---|---|---|
ISO 80000-2 | Quantities and units — Part 2: Mathematics | Governs the notation for matrix powers, the identity matrix and polynomial coefficients, and the typographic distinction between the scalar variable and the matrix argument . |
BLAS Level 3 (xGEMM) | Basic Linear Algebra Subprograms — matrix-matrix operations | Every matrix multiplication in a polynomial evaluation is a GEMM call. Its accumulate form implements one Horner step directly, which is why Horner's scheme maps so cleanly onto optimised libraries. |
IEEE 754-2019 | IEEE Standard for Floating-Point Arithmetic | Defines the overflow and underflow limits that matrix powers reach quickly when the spectral radius departs from one, and the rounding model behind the cancellation seen in expanded-form evaluation. |
ISO/IEC 14882 | Programming languages — C++ | Relevant to the operator-overloading conventions that determine whether A * A means a matrix product or an entrywise product; the Eigen library distinguishes them explicitly with A * A and A.array() * A.array(). |
IEC 61131-3 | Programmable controllers — Programming languages | Applies where a small matrix polynomial — a Rodrigues rotation or a fixed-gain state observer — is evaluated inside deterministic control logic, constraining the arithmetic types and the permissible loop structure. |
Material Selection
For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.
| Representation | Select when | Trade-off |
|---|---|---|
| Exact integer arithmetic | Companion matrices, adjacency matrices and combinatorial walk counting where entries are integers and the answer must be exact. | Exactly correct with no rounding, but the entries of grow geometrically and a fixed-width type will overflow for modest . |
| Exact rational or symbolic coefficients | Deriving an annihilating or characteristic polynomial, or evaluating a polynomial containing a design parameter. | Yields a factorable exact result, but expression size grows rapidly with the degree and the matrix size. |
| IEEE 754 binary64, dense storage | General numerical evaluation of moderate-degree polynomials at matrices of moderate size. | Fast, well supported by BLAS Level 3, but vulnerable to overflow at high powers and to cancellation for non-normal matrices. |
| Sparse storage with matrix-vector application | Very large sparse matrices where only is required, as in Krylov solvers and network centrality measures. | Keeps memory linear in the number of non-zeros and never densifies, but cannot deliver the matrix itself. |
| IEEE 754 binary32 on accelerators | Graphics and robotics kernels such as Rodrigues rotations, evaluated at very high rates and at low degree. | Halves bandwidth and maps well to hardware, but the reduced exponent range makes even modest powers risky and accumulated drift requires periodic re-normalisation. |
| Finite field arithmetic | Coding theory and cryptography, where companion matrices over implement polynomial arithmetic. | No overflow and exact results with fixed operand size, but the factorisation of a polynomial over a finite field differs from its factorisation over . |
Manufacturing Notes
Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.
Operation counts
Naive evaluation of a degree- polynomial by forming each power separately costs matrix multiplications, each about operations. Horner's scheme costs the same number of multiplications but avoids storing the powers. The Paterson-Stockmeyer scheme reduces the count to roughly multiplications by grouping terms, which is the standard technique inside matrix-exponential implementations.
Hand procedure
Compute the powers one at a time, each from the previous, and write them out fully before combining. Scale each by its coefficient, then add entry by entry, taking the diagonal contribution of last so that it is not forgotten. Where the polynomial factors, form the shifted matrices instead — subtracting a constant from the diagonal is far less error-prone than computing a cube.
Library behaviour
numpy.linalg.matrix_power performs true matrix exponentiation by binary powering, whereas A ** 2 on a NumPy array is entrywise. numpy.polyvalfromroots and numpy.polyval are scalar routines and must not be applied to a matrix argument. SymPy evaluates matrix polynomials symbolically and Matrix.charpoly() supplies the characteristic polynomial for degree reduction. scipy.linalg.funm and expm implement matrix functions using polynomial and rational approximations internally.Verification
Three checks are worth running. Compare the expanded and factored evaluations, which exercise entirely different arithmetic. Apply the result to a known eigenvector and confirm the output equals . Finally, check the trace: the trace of equals over the eigenvalues with multiplicity, which is a cheap scalar test of the whole computation.
Numerical stability
Direct summation of an expanded matrix polynomial is stable only when the terms do not vary wildly in magnitude. For non-normal matrices, or for polynomials of high degree, use a Schur decomposition to reduce to triangular form and evaluate there, or apply scaling and squaring so that every polynomial evaluation is performed on a matrix of small norm.
Failure Modes & Common Mistakes
| Failure mode / mistake | Impact | Root cause | Prevention & detection |
|---|---|---|---|
| Adding the constant term as a scalar | high | Writing A + a0 in an array language, which broadcasts the scalar to every entry instead of forming . | Always write explicitly. Unit-test with a diagonal matrix, where the wrong result is immediately visible in the off-diagonal entries. |
| Using entrywise powers instead of matrix powers | high | The ** or .^ operator meaning elementwise exponentiation in NumPy, MATLAB and similar environments. | Use the explicit matrix-power function (numpy.linalg.matrix_power, MATLAB's ^) and verify on a matrix where the two differ, such as one with a zero on the diagonal. |
| Taking to be the zero matrix | medium | Carrying over the intuition that a zero exponent contributes nothing, which deletes the entire constant term. | Fix as a stated convention and check that a constant polynomial evaluates to . |
| Substituting into a non-square matrix | medium | Applying a polynomial to a rectangular matrix, where neither nor is defined. | Assert squareness at the entry point of any routine that evaluates a matrix polynomial. |
| Overflow when forming high powers | high | Computing directly for large when the spectral radius exceeds one, so intermediate entries leave the representable range. | Reduce the polynomial degree using the characteristic polynomial, use Horner's scheme, or scale the matrix before evaluation and undo the scaling afterwards. |
| Catastrophic cancellation in the expanded form | medium | Summing terms of vastly different magnitude for a non-normal matrix, so the significant digits of the answer are lost. | Evaluate through the factored form or a Schur decomposition, and compare the two routes on a representative case. |
| Densifying a sparse matrix | medium | Forming explicitly for a large sparse , where each power fills in more entries until memory is exhausted. | Apply the polynomial to a vector by repeated sparse matrix-vector products rather than forming the polynomial as a matrix. |
| Assuming factors of polynomials in two different matrices commute | low | Over-generalising the commutation property, which holds for polynomials in a single matrix but not for and with . | Check that every factor is a polynomial in the same matrix before reordering. If two matrices are involved, verify explicitly. |
FAQs
Why does the constant term become a multiple of the identity?
Because every term in the sum must be a matrix of the same size for the addition to be defined. The degree-zero term is , and by the same convention that makes for scalars, so the term is . Writing it as a bare scalar is a widely used abbreviation, not a literal instruction.
Does substituting into the factored form really give the same matrix?
Yes, and the reason is commutativity. Every factor is a polynomial in , and any two polynomials in the same matrix commute with one another. Expanding the product therefore behaves exactly as the scalar expansion does, and the factors may be applied in any order without changing the result.
Can I substitute a matrix into a polynomial that involves division?
Not in general. Polynomials are built only from addition, scalar multiplication and powers, all of which square matrices support. Rational functions require inverses, which exist only for non-singular matrices, and even then and differ. Rational matrix functions are studied, but they are a substantially more delicate object.
What does it mean for a polynomial to annihilate a matrix?
It means , the zero matrix. Such polynomials always exist — the Cayley-Hamilton theorem shows the characteristic polynomial is one — and every eigenvalue of must be a root of any annihilating polynomial, because for an eigenvector . That is how factoring an annihilating polynomial exposes eigenvalues.
Why is the factored form often preferred computationally?
Forming each factor requires only subtractions on the diagonal, so no large intermediate coefficients ever appear, and the product needs multiplications, the same as Horner's scheme. The factored form also makes the structure visible: each factor annihilates one eigenspace, which is why a partially applied product has a recognisable rank and column space.
How do I evaluate a very high degree polynomial in a matrix efficiently?
First reduce the degree. By Cayley-Hamilton, is a linear combination of lower powers, so dividing the polynomial by the characteristic polynomial and keeping the remainder leaves an equivalent polynomial of degree less than . Then evaluate the remainder by Horner's scheme, or by the Paterson-Stockmeyer grouping if the degree is still large.
What happens to eigenvalues when a polynomial is applied?
They are mapped through the same polynomial. If then , so the eigenvectors are unchanged and each eigenvalue becomes . This is why forces for every eigenvalue, and why a factor annihilates exactly the eigenspace for .
References
- Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section EE, Subsection PM. Licensed under the GNU Free Documentation License v1.2.
- ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
- Higham, N. J. Functions of Matrices: Theory and Computation. Society for Industrial and Applied Mathematics, 2008.
- Golub, G. H. and Van Loan, C. F. Matrix Computations, 4th edition. Johns Hopkins University Press, 2013.
- Paterson, M. S. and Stockmeyer, L. J. On the Number of Nonscalar Multiplications Necessary to Evaluate Polynomials. SIAM Journal on Computing, 1973.
AI Suggested Questions
- Show how the Paterson-Stockmeyer scheme evaluates a degree-16 matrix polynomial in about eight matrix multiplications instead of fifteen.
- Given a matrix, reduce to a polynomial of degree three using the characteristic polynomial, and verify the reduction numerically.
- Why does hold for the skew-symmetric cross-product matrix of a unit vector, and how does that truncate Rodrigues' rotation formula?
- Construct a non-normal matrix for which the expanded and factored evaluations of the same polynomial differ noticeably in floating point.
- Explain how the residual of GMRES after steps is a matrix polynomial applied to the initial residual, and what that implies about convergence.
- For a sparse adjacency matrix, compare the cost of forming explicitly with applying to a single vector.
Related Calculators
Evaluate any polynomial at a square matrix from coefficients or roots, showing every power and partial sum.
Matrix Power CalculatorRaise a square matrix to an integer power by binary exponentiation, with growth and conditioning diagnostics.
Characteristic Polynomial CalculatorCompute the characteristic polynomial of a matrix and use it to reduce the degree of any polynomial in that matrix.
