← LibraryEquivalent Systems and Equation Operations | KEVOS® MathematicsProject Delivery · Project ManagementLesson 62/189← PrevNext →
ArticlePublished 8 Aug 202622 min readBy Kevin Jogin
Skip to content

Engineering/Mathematics/Systems of Linear Equations

Equivalent Systems and Equation Operations

Two systems of linear equations are equivalent when they have exactly the same solution set, however different they look. Three reversible operations move a system through its equivalence class without disturbing that set, which is the entire licence for solving a hard system by simplifying it into an easy one.

  • Foundation level
  • Stream: linear-systems
  • Reading time 14 min
  • Ref KVS-ENG-MATH-0005
Taxonomy
Engineering / Mathematics
Equivalence means
Identical solution sets, not identical equations
Operations
Swap, scale by α0, add a multiple of one to another
Key theorem
Equation operations preserve the solution set
Why it works
Every operation is reversible
Strategy
Simplify until the answer can be read off, then verify

Overview

Solving a system of linear equations directly is rarely practical beyond two or three unknowns. The workable strategy is indirect: replace the system with a different one that is easier to solve but has the same answers, and then solve that. For the strategy to be sound, same answers must be given a precise meaning and the replacement steps must be proved to preserve it. Those two requirements are what this material supplies.

Two systems are declared equivalent when their solution sets are equal. Nothing else is required — the systems may have different numbers of equations, different coefficients and no visible resemblance. Two equations in two unknowns describing crossing lines, and three equations in the same unknowns describing three lines through the same crossing point, are equivalent by this definition. Equivalence is a deliberately weak relation, and its weakness is exactly what makes it useful.

Three operations generate the equivalence. Any two equations may be exchanged in the list; any equation may be multiplied through by a non-zero scalar; and any multiple of one equation may be added to another, with the source equation left unchanged. These are the equation operations, and the theorem that matters states that applying any of them produces a system equivalent to the original. Applying them in sequence therefore preserves the solution set through arbitrarily long chains of manipulation.

The proof rests on reversibility. Each operation can be undone by another operation of the same kind: a swap is undone by swapping back, a scaling by α is undone by scaling by 1/α, and adding α times equation i to equation j is undone by adding α times equation i to equation j. That symmetry is what lets the argument show set containment in both directions, and it is precisely where the requirement α0 in the scaling operation earns its place.

Definition

Equivalent Systems

ES

Two systems of linear equations in the same unknowns are equivalent when their solution sets are equal. The systems need not have the same number of equations and need not resemble each other in any way; only the sets of n-tuples satisfying them must coincide.

Equivalence is weaker than equality of systems. It is an equivalence relation — reflexive, symmetric and transitive — which is what permits a chain of transformations to be traversed in either direction.

Equation Operations

EO

Three operations may be applied to a system of linear equations:

  • Swap. Exchange the positions of two equations in the list.
  • Scale. Multiply every term of one equation, on both sides, by a non-zero scalar α.
  • Combine. Multiply every term of equation i by a scalar α and add the results, term by term and on both sides, to equation j. Equation i is left unchanged; equation j is replaced.

Only the scaling operation restricts the scalar. In the combining operation α=0 is permitted, because it simply leaves the system unaltered.

Equation Operations Preserve Solution Sets

EOPSS

If any one of the three equation operations is applied to a system of linear equations, the resulting system is equivalent to the original. Consequently any finite sequence of equation operations produces a system with exactly the same solution set as the one it started from.

This theorem is the licence for every elimination method. Without it, simplifying a system would carry no guarantee that the answer obtained belongs to the original problem.

Concepts

Equivalence is about answers, not appearances

Two systems may share not a single coefficient and still be equivalent. Consider two equations in two unknowns whose lines cross at one point, and a second system of three equations whose three lines all pass through that same point. The systems differ in size and in every coefficient, yet both have the solution set {(3,1)} and are therefore equivalent. The relation is a deliberately loose form of sameness: it ignores everything about a system except the set of tuples that satisfy it.

The three operations and their shorthand

Written compactly, the operations are EiEj for a swap, αEi for a scaling with α0, and αEi+Ej for a combination that replaces equation j. The shorthand is worth adopting immediately, because a hand solution is only auditable if each step records which operation was applied to which equations. It also transfers unchanged to the matrix setting, where the same three operations act on rows.

How the preservation theorem is proved

The claim is that two sets are equal, so the standard method applies: show each is a subset of the other. Let S be the solutions of the original system and T those of the transformed one. For the scaling operation, a tuple in S satisfies equation i, so multiplying that equation through by α leaves a true statement and the tuple lies in T; conversely a tuple in T satisfies the scaled equation, and multiplying through by 1/α recovers the original, so the tuple lies in S. The swap case is immediate, since nothing about the order of a list affects which tuples satisfy all of it.

The combining operation and its verification

For the third operation, equation j of the transformed system reads k(αaik+ajk)xk=αbi+bj. A tuple satisfying the original system satisfies equations i and j separately, so the left side evaluates to αbi+bj and the new equation holds. In the reverse direction, a tuple satisfying the transformed system also satisfies equation i — which was not modified — so subtracting α times equation i from the new equation j recovers the original equation j. That subtraction is exactly the inverse operation.

Why α0 for scaling but not for combining

Multiplying an equation by zero replaces it with 0=0, which every tuple satisfies. Information has been destroyed and the operation cannot be undone, so the solution set can grow. In the proof, the reverse inclusion needs multiplication by 1/α, which does not exist when α=0. The combining operation carries no such restriction because the source equation i survives intact: with α=0 nothing changes at all, and for any α the operation is undone by αEi+Ej using the still-available equation i.

The strategy: eliminate, then back-substitute

The operations are applied with a definite goal. Working left to right, use combinations to remove the first unknown from every equation below the first, then the second unknown from every equation below the second, and so on. The result is a staircase or triangular system in which the last equation involves one unknown, the second-to-last two, and so forth. Solving from the bottom up — back-substitution — then determines every unknown. Two special outcomes are informative rather than obstructive: an equation reducing to 0=0 is redundant and can be set aside, while an equation reducing to 0=c with c0 proves the system inconsistent and terminates the analysis immediately.

Solving a system by equation operations

Choose a leading equationSelect an equation in which the first unknown appears. If the first equation lacks it, swap another into place.
Normalise if convenientScale the leading equation by a non-zero constant so its leading coefficient is 1, which keeps the arithmetic clean.
Eliminate belowAdd suitable multiples of the leading equation to the others so the first unknown disappears from all of them.
Repeat on the remainderSet the leading equation aside and apply the same three steps to the reduced set, targeting the next unknown.
Read the terminal rowsAn equation reduced to 0=0 is redundant; one reduced to 0=c with c0 proves inconsistency and ends the analysis.
Back-substitute and verifySolve upward from the last non-trivial equation, then substitute the result into the original system as an independent check.

Equations

Definition of equivalence

EQ-ESEO-01
System PSystem QSP=SQ

Equivalence is equality of solution sets and nothing more. The two systems may differ in size, coefficients and appearance.

The three equation operations

EQ-ESEO-02
EiEj,αEi(α0),αEi+Ej

Swap two equations; scale one equation by a non-zero constant; add a multiple of equation i to equation j, replacing j and leaving i intact.

Effect of the scaling operation on equation i

EQ-ESEO-03
αai1x1+αai2x2++αainxn=αbi

Both sides are multiplied, which is what keeps the equation's truth value unchanged for every tuple when α0.

Effect of the combining operation on equation j

EQ-ESEO-04
(αai1+aj1)x1+(αai2+aj2)x2++(αain+ajn)xn=αbi+bj

Equation j is replaced by the sum of itself and α times equation i. Equation i remains in the system, which is what makes the step reversible.

Key step in the forward inclusion

EQ-ESEO-05
k=1n(αaik+ajk)βk=αk=1naikβk+k=1najkβk=αbi+bj

A tuple β satisfying the original equations i and j automatically satisfies the combined equation. This establishes ST.

Inverse of each operation

EQ-ESEO-06
EiEjEiEj,αEi1αEi,αEi+Ej(α)Ei+Ej

Every operation is undone by an operation of the same type. The middle case is where α0 is indispensable: 1/α must exist.

Terminal forms and what they mean

EQ-ESEO-07
0=0redundant equation,0=c(c0)S=

The two informative degenerate outcomes of elimination. The first reduces the count of independent constraints; the second terminates the problem.

Variable Definitions

Symbols used on this page
SymbolNameMeaningDomain / type
EiEquation iThe i-th equation of the system, treated as an object that operations act on1im
αOperation scalarThe multiplier used in the scaling and combining operationscomplex; non-zero for scaling
aijCoefficientMultiplier of unknown j in equation icomplex number
biConstant termRight-hand side of equation icomplex number
βkCandidate valueValue substituted for unknown xk when testing membership of a solution setcomplex number
SOriginal solution setSolutions of the system before the operationsubset of n
TTransformed solution setSolutions of the system after the operation; the theorem asserts S=Tsubset of n
mEquation countNumber of equations in the systempositive integer
nUnknown countNumber of unknowns in the systempositive integer

Worked Numerical Example

Problem statement

A three-loop instrumentation model produces three constraints on the unknown branch currents x1, x2 and x3, in amperes. The equations arrive from the modeller in an inconvenient order and one of them has been written at double scale. Solve the system using only equation operations, recording each step.

  1. State the system as delivered

    Note that the first equation contains no x1 term and the third has been reported at twice its natural scale. Neither is an obstacle, but both call for an operation before elimination begins.

    0x1+2x2+x3=1x1+2x2+x3=44x1+10x2+8x3=24
  2. Swap to obtain a usable leading equation

    Apply E1E2. Reordering a list of equations cannot change which tuples satisfy all of them, so the new system is equivalent by inspection.

    x1+2x2+x3=42x2+x3=14x1+10x2+8x3=24
  3. Scale the third equation

    Apply 12E3. The scalar 12 is non-zero, so the operation is reversible by 2E3 and the solution set is unchanged. Halving both sides is what keeps the equation's truth value intact.

    x1+2x2+x3=42x2+x3=12x1+5x2+4x3=12
  4. Eliminate x1 from the third equation

    Apply 2E1+E3. Term by term: (22)x1+(54)x2+(42)x3=128. The first equation is untouched, which is what makes the step reversible by 2E1+E3.

    x1+2x2+x3=42x2+x3=1x2+2x3=4
  5. Swap again for a unit leading coefficient

    Apply E2E3 so that the equation with leading coefficient 1 is used to eliminate x2. This is a convenience, not a necessity: scaling would achieve the same effect at the cost of introducing fractions.

    x1+2x2+x3=4x2+2x3=42x2+x3=1
  6. Eliminate x2 from the last equation

    Apply 2E2+E3: (22)x2+(14)x3=18, giving 3x3=9. Then apply 13E3 to normalise.

    x1+2x2+x3=4x2+2x3=4x3=3
  7. Back-substitute

    The third equation forces x3=3. Substituting into the second gives x2=42(3)=2. Substituting both into the first gives x1=42(2)3=5. Every value was forced, so the solution is unique.

    x1=5,x2=2,x3=3
  8. Verify against the original system

    Substitution must be made into the equations as delivered, not into the reduced ones, since an arithmetic slip during elimination produces a chain that is internally consistent but no longer equivalent to the starting point.

    2(2)+3=1,5+2(2)+3=4,4(5)+10(2)+8(3)=24
Result

The branch currents are x1=5 A, x2=2 A and x3=3 A, and the solution is unique because every unknown was forced during back-substitution. All six systems appearing in the chain — the original, the four intermediates and the triangular final form — are equivalent to one another, so the answer read from the last one is the answer to the first. The negative value on x2 indicates flow opposite to the assumed reference direction, not an error.

Applications & Industry Use

Electrical engineering

Hand solution of mesh and nodal equations

Circuit analysis produces small systems that are still routinely solved by hand during design review. Equation operations provide an auditable trail: each step is recorded as a swap, a scaling or a combination, so a reviewer can follow the derivation without redoing it and can locate the exact step at which an error entered.

Structural engineering

Eliminating slave degrees of freedom

Constraint equations tying dependent nodes to master nodes are eliminated from a stiffness system by exactly the combining operation. Because the operation preserves the solution set, the reduced system has the same displacements as the original, and the eliminated quantities are recovered afterwards by substitution.

Chemical engineering

Reconciling material balance equations

Plant balance models frequently contain equations that are combinations of others. Systematic elimination reveals these as rows reducing to 0=0, identifying redundant balances and reducing the count of measurements needed to specify the flowsheet.

Cryptography & coding theory

Solving systems over finite fields

The same three operations apply verbatim when the scalars are drawn from a finite field, which is the setting for syndrome decoding of linear codes and for algebraic attacks. The requirement that the scaling multiplier be non-zero becomes the requirement that it be invertible, which holds for every non-zero element of a field.

Aerospace & navigation

Sequential measurement processing

When navigation measurements arrive one at a time, each is combined into an accumulating triangular system rather than reprocessed from scratch. The combining operation is what merges a new observation into the existing factorisation, and its solution-preserving property is what guarantees the running estimate remains the exact solution of the whole measurement set.

Education & verification

Auditable derivations

Because each operation is named and reversible, a solution written as a chain of operations can be checked forwards or backwards. Applying the inverse operations to the final system must reconstruct the original exactly, which is a stronger check than merely substituting the answer and one that localises the step where any discrepancy arose.

Design Considerations

Never multiply an equation by zero

Scaling by zero replaces an equation with 0=0, destroying a constraint and enlarging the solution set. It is the only one of the three operations that can produce a non-equivalent system, and it is prohibited for exactly that reason. In code the guard belongs at the point of use, not in a comment.

Keep the source equation intact when combining

The combining operation replaces the target equation and leaves the source unchanged. Replacing both, or using an equation that has already been modified in the same sweep, breaks reversibility and can silently change the solution set. Fixing the source equation for the duration of a sweep is the single most effective discipline in hand elimination.

Choose the pivot equation for arithmetic convenience

The final answer does not depend on which equation is used to eliminate an unknown, so the choice can be made purely for convenience. In exact arithmetic, prefer an equation whose leading coefficient is 1 or 1 to avoid fractions; a swap costs nothing and often saves a page of working.

In floating point, choose the pivot for stability instead

When the arithmetic is inexact, dividing by a small leading coefficient amplifies rounding error in every subsequent step. The standard remedy is partial pivoting: swap in the equation with the largest available leading coefficient in magnitude. This is a numerical choice with no effect on the exact answer, and it should be applied automatically rather than case by case.

Record every operation

A sequence of systems with no annotation is unverifiable. Recording each step in the form αEi+Ej allows the chain to be replayed, reversed and reviewed, and it is the only practical way to find the step where an arithmetic error entered. In software the equivalent is retaining the multipliers, which is what makes an LU factorisation reusable.

Verify against the original, not the reduced, system

An arithmetic slip during elimination produces a system that is perfectly self-consistent but no longer equivalent to the one posed. Substituting the answer back into the reduced system will confirm it happily. Only substitution into the equations as originally stated tests the whole chain.

Standards & Codes

Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.

Applicable standards, conventions and reference implementations
ReferenceTitleRelevance to this topic
ISO 80000-2Quantities and units — Part 2: MathematicsFixes the notation for set equality, subset relations and indexed equations used in stating and proving the solution-preservation theorem.
IEEE 754-2019IEEE Standard for Floating-Point ArithmeticGoverns the arithmetic of the scaling and combining operations in practice, and explains why an operation that is exactly reversible in theory is only approximately reversible in a computer.
LAPACK / BLAS referenceLinear Algebra PACKage reference implementationImplements the combining operation as the rank-one update at the heart of xGETRF, and the swap operation as the row interchange recorded in the pivot vector.
BLAS Level 1Basic Linear Algebra Subprograms, vector operationsProvides the exact primitives the operations require: xSCAL for scaling an equation, xAXPY for adding a multiple of one to another, and xSWAP for the interchange.
ISO/IEC 40314Mathematical Markup Language (MathML) Version 3.0Encodes the aligned systems and operation annotations on this page semantically, so each step of a derivation is machine-readable rather than an image.

Material Selection

For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.

Representation and precision selection
RepresentationSelect whenTrade-off
Exact rational arithmeticHand-checkable derivations, symbolic work, and any case where the reversibility of the chain must be provable.Every operation is exactly reversible and the equivalence claim is literal, but denominators proliferate during elimination unless pivots are chosen carefully.
Integer arithmetic with fraction-free eliminationInteger-coefficient systems where fractions are to be avoided entirely.Keeps all intermediates integral by scaling before combining, at the cost of larger operands and a form that is echelon rather than fully reduced.
IEEE 754 binary64The default for any system assembled from engineering data.Ample precision and speed, but operations are only approximately reversible, so a long chain accumulates rounding and the reduced system is not exactly equivalent to the original.
Finite field arithmeticCoding theory, cryptanalysis and modular rank certification.Exact, with no operand growth and every non-zero scalar invertible, but the pivot pattern over a finite field may differ from the pattern over the rationals.
Fixed-point arithmeticEmbedded solvers on hardware without a floating-point unit.Deterministic and cheap, but the scaling operation can overflow, so the multiplier range must be bounded by analysis before deployment.
Deferred multiplier storageThe same coefficient system will be solved repeatedly with different constants.Storing the combining multipliers instead of applying them to the constants turns the elimination into a reusable factorisation, at the cost of extra memory and a two-stage solve.

Manufacturing Notes

Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.

Operation count for elimination

Reducing an n×n system to triangular form by the combining operation costs about 23n3 floating-point operations, dominated by the innermost multiply-and-add. Back-substitution then costs about n2. The cubic term is why doubling the number of unknowns multiplies the work by roughly eight, a scaling that governs every practical decision about problem size.

Hand procedure that avoids the common error

Fix the pivot equation, compute every multiplier against it first, and only then apply all the combinations. Applying one combination and then computing the next multiplier from an equation that has already been altered is the dominant source of hand-computation error, and it is eliminated entirely by this ordering.

Swaps cost nothing but must be recorded

In software a swap is normally implemented as an index permutation rather than a data movement, so its arithmetic cost is zero. What matters is that the permutation is stored: LAPACK returns it in an integer pivot array, and any solution computed from the factors must apply the same permutation to the constants before back-substitution.

Reversibility is only approximate in floating point

The scaling and combining operations are exactly invertible over the rationals. In binary64 each introduces a relative error of order 1016, so applying an operation and then its inverse does not return the original coefficients bit for bit. Over a long chain these errors accumulate, which is why growth factors and condition numbers, rather than the exact equivalence theorem, govern the accuracy of a computed answer.

Verifying a chain of operations

Two independent checks are worth the effort. First, substitute the computed answer into the original equations and record the largest residual. Second, apply the inverse operations in reverse order to the final system and confirm the original coefficients are recovered. The second check localises an error to a specific step, which substitution alone cannot do.

Failure Modes & Common Mistakes

Failure modes, root causes and prevention
Failure mode / mistakeImpactRoot causePrevention & detection
Scaling an equation by zerohighA multiplier computed from data happens to be zero, and the guard on the scaling operation is missing.Test the multiplier before applying the operation; scaling is the only equation operation with a precondition, and it must be enforced.
Modifying the source equation when combininghighBoth the source and the target are replaced, so the step is no longer reversible and information is lost.Write the operation as αEi+Ej with the explicit convention that only Ej changes, and preserve Ei untouched through the sweep.
Using an already-modified equation as the pivothighMultipliers for later eliminations are computed from an equation that has been altered earlier in the same sweep.Fix the pivot equation before the sweep begins and compute all multipliers against its original coefficients.
Applying column operations by analogyhighSwapping or combining the coefficients of two unknowns, by analogy with the permitted operations on equations.Operations act on equations only. Manipulating unknowns relabels the problem and does not preserve the solution set as stated.
Forgetting to scale the constant termhighAn equation is multiplied on the left only, so the equality no longer holds.Treat the constant as the final term of the equation and apply every operation across the whole row including it.
Verifying against the reduced systemmediumThe computed answer is substituted into the final simplified equations, which it satisfies even when the chain contains an arithmetic error.Always substitute into the equations as originally posed and report the largest residual across all of them.
Discarding a redundant equation carelesslymediumAn equation reducing to 0=0 is deleted without recording that the count of independent constraints has dropped.Note each redundancy explicitly, since the count of remaining independent equations is what determines the degrees of freedom in the answer.
Dividing by a small pivot in floating pointmediumThe leading coefficient of the pivot equation is nearly zero, so the multipliers are enormous and rounding is amplified.Apply partial pivoting: swap in the equation with the largest leading coefficient in magnitude before eliminating.

FAQs

What exactly makes two systems equivalent?

Equality of their solution sets, and nothing else. The systems may have different numbers of equations and share no coefficients. Two equations describing crossing lines and three equations describing three lines through the same point are equivalent, because both have the same single solution. Equivalence deliberately discards everything about a system except which tuples satisfy it.

Why must the scaling multiplier be non-zero?

Because multiplying an equation by zero turns it into 0=0, which every tuple satisfies. A constraint has been erased and the solution set can grow, so the transformed system is not equivalent. The proof makes the requirement visible: the reverse inclusion needs multiplication by 1/α, which does not exist when α=0.

Why is there no such restriction on the combining operation?

Because the source equation survives. Adding zero times equation i to equation j leaves the system exactly as it was, so nothing is lost. For any α, the operation is undone by adding α times equation i to equation j, and equation i is still there to do it with. Reversibility never fails, so no precondition is needed.

Does it matter which equation I use as the pivot?

Not for the answer, which is determined by the system rather than by the route taken. It matters considerably for the effort: choosing an equation with a leading coefficient of 1 avoids fractions in exact arithmetic. In floating point it matters for accuracy, where the standard rule is to pivot on the largest available coefficient in magnitude.

What does it mean when an equation reduces to 0=0?

That the equation was a combination of the others and imposed no independent constraint. It can be set aside, but the fact should be recorded, because the count of independent equations is what determines how many degrees of freedom the solution set has. A redundant equation is a common and entirely benign outcome of a model with duplicated information.

And when an equation reduces to 0=5?

The system is inconsistent and its solution set is empty. Since every system in the chain is equivalent to the original, the contradiction belongs to the original system too. The analysis can stop immediately — there is no need to continue reducing — and the combination of original equations that produced the contradictory row is the most useful diagnostic available.

Can I apply these operations to the columns instead of the equations?

No. The three operations are defined on equations, and the preservation theorem is proved for equations only. Manipulating columns rearranges or mixes the unknowns themselves, which changes what the tuples mean rather than which tuples satisfy the system. Every reliable elimination method restricts itself to operations on equations, or equivalently on rows of the augmented matrix.

References

  1. Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section SSLE, Subsection ESEO; Theorem EOPSS. Licensed under the GNU Free Documentation License v1.2.
  2. ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
  3. Golub, G. H. and Van Loan, C. F. Matrix Computations, 4th edition. Johns Hopkins University Press, 2013.
  4. Higham, N. J. Accuracy and Stability of Numerical Algorithms, 2nd edition. Society for Industrial and Applied Mathematics, 2002.
  5. Anderson, E. et al. LAPACK Users' Guide, 3rd edition. Society for Industrial and Applied Mathematics, 1999.

AI Suggested Questions

  • Show me a sequence of equation operations that transforms one given system into another, proving they are equivalent.
  • What goes wrong, step by step, in the proof of the preservation theorem if the scaling multiplier is allowed to be zero?
  • Compare the growth of intermediate fractions when I pivot on the largest coefficient versus the first non-zero one, for the same system.
  • Take a system whose elimination produces 0=0 and identify exactly which original equations were combined to create it.
  • Explain how the combining operation corresponds to multiplying by an elementary matrix, and what the inverse of that matrix is.
  • Demonstrate a case where floating-point rounding makes an equation operation non-reversible, and quantify the resulting error in the answer.

Related Calculators

Continue learning

Algebraic and Geometric Multiplicities of Eigenvalues | KEVOS® MathematicsArticle · Project ManagementAmitsur’s Theorem on the Radical of a Polynomial Ring | KEVOS®Article · Project ManagementAmitsur’s Theorem on the Radical of an Algebra of Small Dimension | KEVOS®Article · Project ManagementArchetypes: Reference Catalogue of Worked Systems | KEVOS® MathematicsArticle · Project Management