Engineering/Mathematics/Systems of Linear Equations
Equivalent Systems and Equation Operations
Two systems of linear equations are equivalent when they have exactly the same solution set, however different they look. Three reversible operations move a system through its equivalence class without disturbing that set, which is the entire licence for solving a hard system by simplifying it into an easy one.
- Foundation level
- Stream: linear-systems
- Reading time 14 min
- Ref KVS-ENG-MATH-0005
- Taxonomy
- Engineering / Mathematics
- Equivalence means
- Identical solution sets, not identical equations
- Operations
- Swap, scale by , add a multiple of one to another
- Key theorem
- Equation operations preserve the solution set
- Why it works
- Every operation is reversible
- Strategy
- Simplify until the answer can be read off, then verify
Overview
Solving a system of linear equations directly is rarely practical beyond two or three unknowns. The workable strategy is indirect: replace the system with a different one that is easier to solve but has the same answers, and then solve that. For the strategy to be sound, same answers must be given a precise meaning and the replacement steps must be proved to preserve it. Those two requirements are what this material supplies.
Two systems are declared equivalent when their solution sets are equal. Nothing else is required — the systems may have different numbers of equations, different coefficients and no visible resemblance. Two equations in two unknowns describing crossing lines, and three equations in the same unknowns describing three lines through the same crossing point, are equivalent by this definition. Equivalence is a deliberately weak relation, and its weakness is exactly what makes it useful.
Three operations generate the equivalence. Any two equations may be exchanged in the list; any equation may be multiplied through by a non-zero scalar; and any multiple of one equation may be added to another, with the source equation left unchanged. These are the equation operations, and the theorem that matters states that applying any of them produces a system equivalent to the original. Applying them in sequence therefore preserves the solution set through arbitrarily long chains of manipulation.
The proof rests on reversibility. Each operation can be undone by another operation of the same kind: a swap is undone by swapping back, a scaling by is undone by scaling by , and adding times equation to equation is undone by adding times equation to equation . That symmetry is what lets the argument show set containment in both directions, and it is precisely where the requirement in the scaling operation earns its place.
Definition
Equivalent Systems
ESTwo systems of linear equations in the same unknowns are equivalent when their solution sets are equal. The systems need not have the same number of equations and need not resemble each other in any way; only the sets of -tuples satisfying them must coincide.
Equivalence is weaker than equality of systems. It is an equivalence relation — reflexive, symmetric and transitive — which is what permits a chain of transformations to be traversed in either direction.
Equation Operations
EOThree operations may be applied to a system of linear equations:
- Swap. Exchange the positions of two equations in the list.
- Scale. Multiply every term of one equation, on both sides, by a non-zero scalar .
- Combine. Multiply every term of equation by a scalar and add the results, term by term and on both sides, to equation . Equation is left unchanged; equation is replaced.
Only the scaling operation restricts the scalar. In the combining operation is permitted, because it simply leaves the system unaltered.
Equation Operations Preserve Solution Sets
EOPSSIf any one of the three equation operations is applied to a system of linear equations, the resulting system is equivalent to the original. Consequently any finite sequence of equation operations produces a system with exactly the same solution set as the one it started from.
This theorem is the licence for every elimination method. Without it, simplifying a system would carry no guarantee that the answer obtained belongs to the original problem.
Concepts
Equivalence is about answers, not appearances
Two systems may share not a single coefficient and still be equivalent. Consider two equations in two unknowns whose lines cross at one point, and a second system of three equations whose three lines all pass through that same point. The systems differ in size and in every coefficient, yet both have the solution set and are therefore equivalent. The relation is a deliberately loose form of sameness: it ignores everything about a system except the set of tuples that satisfy it.
The three operations and their shorthand
Written compactly, the operations are for a swap, for a scaling with , and for a combination that replaces equation . The shorthand is worth adopting immediately, because a hand solution is only auditable if each step records which operation was applied to which equations. It also transfers unchanged to the matrix setting, where the same three operations act on rows.
How the preservation theorem is proved
The claim is that two sets are equal, so the standard method applies: show each is a subset of the other. Let be the solutions of the original system and those of the transformed one. For the scaling operation, a tuple in satisfies equation , so multiplying that equation through by leaves a true statement and the tuple lies in ; conversely a tuple in satisfies the scaled equation, and multiplying through by recovers the original, so the tuple lies in . The swap case is immediate, since nothing about the order of a list affects which tuples satisfy all of it.
The combining operation and its verification
For the third operation, equation of the transformed system reads . A tuple satisfying the original system satisfies equations and separately, so the left side evaluates to and the new equation holds. In the reverse direction, a tuple satisfying the transformed system also satisfies equation — which was not modified — so subtracting times equation from the new equation recovers the original equation . That subtraction is exactly the inverse operation.
Why for scaling but not for combining
Multiplying an equation by zero replaces it with , which every tuple satisfies. Information has been destroyed and the operation cannot be undone, so the solution set can grow. In the proof, the reverse inclusion needs multiplication by , which does not exist when . The combining operation carries no such restriction because the source equation survives intact: with nothing changes at all, and for any the operation is undone by using the still-available equation .
The strategy: eliminate, then back-substitute
The operations are applied with a definite goal. Working left to right, use combinations to remove the first unknown from every equation below the first, then the second unknown from every equation below the second, and so on. The result is a staircase or triangular system in which the last equation involves one unknown, the second-to-last two, and so forth. Solving from the bottom up — back-substitution — then determines every unknown. Two special outcomes are informative rather than obstructive: an equation reducing to is redundant and can be set aside, while an equation reducing to with proves the system inconsistent and terminates the analysis immediately.
Solving a system by equation operations
Equations
Definition of equivalence
EQ-ESEO-01Equivalence is equality of solution sets and nothing more. The two systems may differ in size, coefficients and appearance.
The three equation operations
EQ-ESEO-02Swap two equations; scale one equation by a non-zero constant; add a multiple of equation to equation , replacing and leaving intact.
Effect of the scaling operation on equation
EQ-ESEO-03Both sides are multiplied, which is what keeps the equation's truth value unchanged for every tuple when .
Effect of the combining operation on equation
EQ-ESEO-04Equation is replaced by the sum of itself and times equation . Equation remains in the system, which is what makes the step reversible.
Key step in the forward inclusion
EQ-ESEO-05A tuple satisfying the original equations and automatically satisfies the combined equation. This establishes .
Inverse of each operation
EQ-ESEO-06Every operation is undone by an operation of the same type. The middle case is where is indispensable: must exist.
Terminal forms and what they mean
EQ-ESEO-07The two informative degenerate outcomes of elimination. The first reduces the count of independent constraints; the second terminates the problem.
Variable Definitions
| Symbol | Name | Meaning | Domain / type |
|---|---|---|---|
| Equation | The -th equation of the system, treated as an object that operations act on | ||
| Operation scalar | The multiplier used in the scaling and combining operations | complex; non-zero for scaling | |
| Coefficient | Multiplier of unknown in equation | complex number | |
| Constant term | Right-hand side of equation | complex number | |
| Candidate value | Value substituted for unknown when testing membership of a solution set | complex number | |
| Original solution set | Solutions of the system before the operation | subset of | |
| Transformed solution set | Solutions of the system after the operation; the theorem asserts | subset of | |
| Equation count | Number of equations in the system | positive integer | |
| Unknown count | Number of unknowns in the system | positive integer |
Worked Numerical Example
Problem statement
A three-loop instrumentation model produces three constraints on the unknown branch currents , and , in amperes. The equations arrive from the modeller in an inconvenient order and one of them has been written at double scale. Solve the system using only equation operations, recording each step.
State the system as delivered
Note that the first equation contains no term and the third has been reported at twice its natural scale. Neither is an obstacle, but both call for an operation before elimination begins.
Swap to obtain a usable leading equation
Apply . Reordering a list of equations cannot change which tuples satisfy all of them, so the new system is equivalent by inspection.
Scale the third equation
Apply . The scalar is non-zero, so the operation is reversible by and the solution set is unchanged. Halving both sides is what keeps the equation's truth value intact.
Eliminate from the third equation
Apply . Term by term: . The first equation is untouched, which is what makes the step reversible by .
Swap again for a unit leading coefficient
Apply so that the equation with leading coefficient is used to eliminate . This is a convenience, not a necessity: scaling would achieve the same effect at the cost of introducing fractions.
Eliminate from the last equation
Apply : , giving . Then apply to normalise.
Back-substitute
The third equation forces . Substituting into the second gives . Substituting both into the first gives . Every value was forced, so the solution is unique.
Verify against the original system
Substitution must be made into the equations as delivered, not into the reduced ones, since an arithmetic slip during elimination produces a chain that is internally consistent but no longer equivalent to the starting point.
The branch currents are A, A and A, and the solution is unique because every unknown was forced during back-substitution. All six systems appearing in the chain — the original, the four intermediates and the triangular final form — are equivalent to one another, so the answer read from the last one is the answer to the first. The negative value on indicates flow opposite to the assumed reference direction, not an error.
Applications & Industry Use
Hand solution of mesh and nodal equations
Circuit analysis produces small systems that are still routinely solved by hand during design review. Equation operations provide an auditable trail: each step is recorded as a swap, a scaling or a combination, so a reviewer can follow the derivation without redoing it and can locate the exact step at which an error entered.
Eliminating slave degrees of freedom
Constraint equations tying dependent nodes to master nodes are eliminated from a stiffness system by exactly the combining operation. Because the operation preserves the solution set, the reduced system has the same displacements as the original, and the eliminated quantities are recovered afterwards by substitution.
Reconciling material balance equations
Plant balance models frequently contain equations that are combinations of others. Systematic elimination reveals these as rows reducing to , identifying redundant balances and reducing the count of measurements needed to specify the flowsheet.
Solving systems over finite fields
The same three operations apply verbatim when the scalars are drawn from a finite field, which is the setting for syndrome decoding of linear codes and for algebraic attacks. The requirement that the scaling multiplier be non-zero becomes the requirement that it be invertible, which holds for every non-zero element of a field.
Sequential measurement processing
When navigation measurements arrive one at a time, each is combined into an accumulating triangular system rather than reprocessed from scratch. The combining operation is what merges a new observation into the existing factorisation, and its solution-preserving property is what guarantees the running estimate remains the exact solution of the whole measurement set.
Auditable derivations
Because each operation is named and reversible, a solution written as a chain of operations can be checked forwards or backwards. Applying the inverse operations to the final system must reconstruct the original exactly, which is a stronger check than merely substituting the answer and one that localises the step where any discrepancy arose.
Design Considerations
Never multiply an equation by zero
Scaling by zero replaces an equation with , destroying a constraint and enlarging the solution set. It is the only one of the three operations that can produce a non-equivalent system, and it is prohibited for exactly that reason. In code the guard belongs at the point of use, not in a comment.
Keep the source equation intact when combining
The combining operation replaces the target equation and leaves the source unchanged. Replacing both, or using an equation that has already been modified in the same sweep, breaks reversibility and can silently change the solution set. Fixing the source equation for the duration of a sweep is the single most effective discipline in hand elimination.
Choose the pivot equation for arithmetic convenience
The final answer does not depend on which equation is used to eliminate an unknown, so the choice can be made purely for convenience. In exact arithmetic, prefer an equation whose leading coefficient is or to avoid fractions; a swap costs nothing and often saves a page of working.
In floating point, choose the pivot for stability instead
When the arithmetic is inexact, dividing by a small leading coefficient amplifies rounding error in every subsequent step. The standard remedy is partial pivoting: swap in the equation with the largest available leading coefficient in magnitude. This is a numerical choice with no effect on the exact answer, and it should be applied automatically rather than case by case.
Record every operation
A sequence of systems with no annotation is unverifiable. Recording each step in the form allows the chain to be replayed, reversed and reviewed, and it is the only practical way to find the step where an arithmetic error entered. In software the equivalent is retaining the multipliers, which is what makes an LU factorisation reusable.
Verify against the original, not the reduced, system
An arithmetic slip during elimination produces a system that is perfectly self-consistent but no longer equivalent to the one posed. Substituting the answer back into the reduced system will confirm it happily. Only substitution into the equations as originally stated tests the whole chain.
Standards & Codes
Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.
| Reference | Title | Relevance to this topic |
|---|---|---|
ISO 80000-2 | Quantities and units — Part 2: Mathematics | Fixes the notation for set equality, subset relations and indexed equations used in stating and proving the solution-preservation theorem. |
IEEE 754-2019 | IEEE Standard for Floating-Point Arithmetic | Governs the arithmetic of the scaling and combining operations in practice, and explains why an operation that is exactly reversible in theory is only approximately reversible in a computer. |
LAPACK / BLAS reference | Linear Algebra PACKage reference implementation | Implements the combining operation as the rank-one update at the heart of xGETRF, and the swap operation as the row interchange recorded in the pivot vector. |
BLAS Level 1 | Basic Linear Algebra Subprograms, vector operations | Provides the exact primitives the operations require: xSCAL for scaling an equation, xAXPY for adding a multiple of one to another, and xSWAP for the interchange. |
ISO/IEC 40314 | Mathematical Markup Language (MathML) Version 3.0 | Encodes the aligned systems and operation annotations on this page semantically, so each step of a derivation is machine-readable rather than an image. |
Material Selection
For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.
| Representation | Select when | Trade-off |
|---|---|---|
| Exact rational arithmetic | Hand-checkable derivations, symbolic work, and any case where the reversibility of the chain must be provable. | Every operation is exactly reversible and the equivalence claim is literal, but denominators proliferate during elimination unless pivots are chosen carefully. |
| Integer arithmetic with fraction-free elimination | Integer-coefficient systems where fractions are to be avoided entirely. | Keeps all intermediates integral by scaling before combining, at the cost of larger operands and a form that is echelon rather than fully reduced. |
| IEEE 754 binary64 | The default for any system assembled from engineering data. | Ample precision and speed, but operations are only approximately reversible, so a long chain accumulates rounding and the reduced system is not exactly equivalent to the original. |
| Finite field arithmetic | Coding theory, cryptanalysis and modular rank certification. | Exact, with no operand growth and every non-zero scalar invertible, but the pivot pattern over a finite field may differ from the pattern over the rationals. |
| Fixed-point arithmetic | Embedded solvers on hardware without a floating-point unit. | Deterministic and cheap, but the scaling operation can overflow, so the multiplier range must be bounded by analysis before deployment. |
| Deferred multiplier storage | The same coefficient system will be solved repeatedly with different constants. | Storing the combining multipliers instead of applying them to the constants turns the elimination into a reusable factorisation, at the cost of extra memory and a two-stage solve. |
Manufacturing Notes
Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.
Operation count for elimination
Reducing an system to triangular form by the combining operation costs about floating-point operations, dominated by the innermost multiply-and-add. Back-substitution then costs about . The cubic term is why doubling the number of unknowns multiplies the work by roughly eight, a scaling that governs every practical decision about problem size.
Hand procedure that avoids the common error
Fix the pivot equation, compute every multiplier against it first, and only then apply all the combinations. Applying one combination and then computing the next multiplier from an equation that has already been altered is the dominant source of hand-computation error, and it is eliminated entirely by this ordering.
Swaps cost nothing but must be recorded
In software a swap is normally implemented as an index permutation rather than a data movement, so its arithmetic cost is zero. What matters is that the permutation is stored: LAPACK returns it in an integer pivot array, and any solution computed from the factors must apply the same permutation to the constants before back-substitution.
Reversibility is only approximate in floating point
The scaling and combining operations are exactly invertible over the rationals. In binary64 each introduces a relative error of order , so applying an operation and then its inverse does not return the original coefficients bit for bit. Over a long chain these errors accumulate, which is why growth factors and condition numbers, rather than the exact equivalence theorem, govern the accuracy of a computed answer.
Verifying a chain of operations
Two independent checks are worth the effort. First, substitute the computed answer into the original equations and record the largest residual. Second, apply the inverse operations in reverse order to the final system and confirm the original coefficients are recovered. The second check localises an error to a specific step, which substitution alone cannot do.
Failure Modes & Common Mistakes
| Failure mode / mistake | Impact | Root cause | Prevention & detection |
|---|---|---|---|
| Scaling an equation by zero | high | A multiplier computed from data happens to be zero, and the guard on the scaling operation is missing. | Test the multiplier before applying the operation; scaling is the only equation operation with a precondition, and it must be enforced. |
| Modifying the source equation when combining | high | Both the source and the target are replaced, so the step is no longer reversible and information is lost. | Write the operation as with the explicit convention that only changes, and preserve untouched through the sweep. |
| Using an already-modified equation as the pivot | high | Multipliers for later eliminations are computed from an equation that has been altered earlier in the same sweep. | Fix the pivot equation before the sweep begins and compute all multipliers against its original coefficients. |
| Applying column operations by analogy | high | Swapping or combining the coefficients of two unknowns, by analogy with the permitted operations on equations. | Operations act on equations only. Manipulating unknowns relabels the problem and does not preserve the solution set as stated. |
| Forgetting to scale the constant term | high | An equation is multiplied on the left only, so the equality no longer holds. | Treat the constant as the final term of the equation and apply every operation across the whole row including it. |
| Verifying against the reduced system | medium | The computed answer is substituted into the final simplified equations, which it satisfies even when the chain contains an arithmetic error. | Always substitute into the equations as originally posed and report the largest residual across all of them. |
| Discarding a redundant equation carelessly | medium | An equation reducing to is deleted without recording that the count of independent constraints has dropped. | Note each redundancy explicitly, since the count of remaining independent equations is what determines the degrees of freedom in the answer. |
| Dividing by a small pivot in floating point | medium | The leading coefficient of the pivot equation is nearly zero, so the multipliers are enormous and rounding is amplified. | Apply partial pivoting: swap in the equation with the largest leading coefficient in magnitude before eliminating. |
FAQs
What exactly makes two systems equivalent?
Equality of their solution sets, and nothing else. The systems may have different numbers of equations and share no coefficients. Two equations describing crossing lines and three equations describing three lines through the same point are equivalent, because both have the same single solution. Equivalence deliberately discards everything about a system except which tuples satisfy it.
Why must the scaling multiplier be non-zero?
Because multiplying an equation by zero turns it into , which every tuple satisfies. A constraint has been erased and the solution set can grow, so the transformed system is not equivalent. The proof makes the requirement visible: the reverse inclusion needs multiplication by , which does not exist when .
Why is there no such restriction on the combining operation?
Because the source equation survives. Adding zero times equation to equation leaves the system exactly as it was, so nothing is lost. For any , the operation is undone by adding times equation to equation , and equation is still there to do it with. Reversibility never fails, so no precondition is needed.
Does it matter which equation I use as the pivot?
Not for the answer, which is determined by the system rather than by the route taken. It matters considerably for the effort: choosing an equation with a leading coefficient of avoids fractions in exact arithmetic. In floating point it matters for accuracy, where the standard rule is to pivot on the largest available coefficient in magnitude.
What does it mean when an equation reduces to ?
That the equation was a combination of the others and imposed no independent constraint. It can be set aside, but the fact should be recorded, because the count of independent equations is what determines how many degrees of freedom the solution set has. A redundant equation is a common and entirely benign outcome of a model with duplicated information.
And when an equation reduces to ?
The system is inconsistent and its solution set is empty. Since every system in the chain is equivalent to the original, the contradiction belongs to the original system too. The analysis can stop immediately — there is no need to continue reducing — and the combination of original equations that produced the contradictory row is the most useful diagnostic available.
Can I apply these operations to the columns instead of the equations?
No. The three operations are defined on equations, and the preservation theorem is proved for equations only. Manipulating columns rearranges or mixes the unknowns themselves, which changes what the tuples mean rather than which tuples satisfy the system. Every reliable elimination method restricts itself to operations on equations, or equivalently on rows of the augmented matrix.
References
- Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section SSLE, Subsection ESEO; Theorem EOPSS. Licensed under the GNU Free Documentation License v1.2.
- ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
- Golub, G. H. and Van Loan, C. F. Matrix Computations, 4th edition. Johns Hopkins University Press, 2013.
- Higham, N. J. Accuracy and Stability of Numerical Algorithms, 2nd edition. Society for Industrial and Applied Mathematics, 2002.
- Anderson, E. et al. LAPACK Users' Guide, 3rd edition. Society for Industrial and Applied Mathematics, 1999.
AI Suggested Questions
- Show me a sequence of equation operations that transforms one given system into another, proving they are equivalent.
- What goes wrong, step by step, in the proof of the preservation theorem if the scaling multiplier is allowed to be zero?
- Compare the growth of intermediate fractions when I pivot on the largest coefficient versus the first non-zero one, for the same system.
- Take a system whose elimination produces and identify exactly which original equations were combined to create it.
- Explain how the combining operation corresponds to multiplying by an elementary matrix, and what the inverse of that matrix is.
- Demonstrate a case where floating-point rounding makes an equation operation non-reversible, and quantify the resulting error in the answer.
Related Calculators
Apply swaps, scalings and combinations to a system step by step and see the solution set confirmed unchanged at each stage.
Linear System SolverSolve and report consistency, rank and the vector form of the solution set.
Elimination TraceGenerate a fully annotated elimination sequence for a given system, with every multiplier and pivot choice recorded.
