← LibraryWhat Is Linear Algebra? | KEVOS® MathematicsProject Delivery · Project ManagementLesson 188/189← PrevNext →
ArticlePublished 8 Aug 202622 min readBy Kevin Jogin
Skip to content

Engineering/Mathematics/Systems of Linear Equations

What Is Linear Algebra?

Linear algebra is the algebra of objects built from addition and scalar multiplication alone, and of the flat sets those two operations produce. That deliberate restriction is what makes the subject completely solvable, and it is why almost every quantitative discipline reduces its hardest problems to linear ones.

  • Foundation level
  • Stream: foundations
  • Reading time 12 min
  • Ref KVS-ENG-MATH-0001
Taxonomy
Engineering / Mathematics
Permitted operations
Addition and multiplication by a constant
Geometric objects
Points, lines, planes and their higher-dimensional analogues
Scalar field
Real numbers or complex numbers
Central problem
Solving m linear equations in n unknowns
Level
Foundation

Overview

The name of the subject states its two commitments. Linear means flat or straight: the sets of points that appear as solutions are lines, planes and their higher-dimensional counterparts, never curves or surfaces with genuine curvature. Algebra means a collection of objects together with operations obeying stated rules. Linear algebra is therefore the study of a specific algebraic system — vectors, matrices and the transformations between them — whose defining restriction is that only two operations are permitted: adding two objects, and scaling an object by a constant.

That restriction looks severe and is in fact the entire point. Because only addition and scaling are allowed, an unknown never appears squared, never multiplies another unknown, and never sits inside a transcendental function. The consequence is decisive: the solution set of any linear system can be described completely and found by a finite, mechanical procedure. No comparable statement holds for polynomial or transcendental systems, where existence, uniqueness and computation are all open-ended.

The geometry is genuinely helpful in two and three dimensions. A single equation ax+by=c describes a line in the plane; ax+by+cz=d describes a plane in space; solving two equations simultaneously means intersecting two such flat sets. But most engineering problems live in dozens, thousands or millions of variables, where visualisation fails entirely. The algebraic machinery does not care. It manipulates flat objects in n with the same procedures used for n=2, and this dimension-independence is the practical reason linear algebra is the workhorse of computation.

Two further points shape everything that follows. First, linearity is a property of an equation relative to a nominated set of unknowns: the same expression can be nonlinear in one variable and perfectly linear in another, which is exactly why curve fitting, filter design and finite element analysis all reduce to linear systems. Second, the scalars are taken from rather than from the outset. Real arithmetic suffices for a long time, but eigenvalues, oscillatory modes and signal phase all force complex scalars eventually, and building them in early avoids rewriting the theory later.

Definition

Linear Equation

An equation in the unknowns x1,x2,,xn is linear when it can be written as a1x1+a2x2++anxn=b, where the coefficients a1,,an and the constant b do not depend on the unknowns. Every unknown appears to the first power, alone, multiplied only by a constant, and the resulting terms are combined only by addition.

Subtraction is addition of a negated term and division by a constant is multiplication by its reciprocal, so both are permitted. Division by an unknown is not.

Linearity (relative to nominated unknowns)

Linearity is not a property of an expression in isolation but of an expression together with a declared list of unknowns. The relation y=c0+c1t+c2t2 is nonlinear in t and simultaneously linear in c0,c1,c2. Declaring which symbols are unknown and which are data is therefore the first step in any modelling exercise, and it determines whether linear algebra applies.

Affine Equation

An equation of the form a1x1++anxn=b with b0 is affine rather than strictly linear: its solution set is a flat set that does not pass through the origin, and it does not satisfy L(αu)=αL(u). The term linear equation is used for both cases by long-standing convention; the distinction matters when the equation is reinterpreted as a function.

Concepts

Flatness is the geometric signature of linearity

In the plane, the solution set of one linear equation in two unknowns is a straight line; in space, one linear equation in three unknowns gives a plane, and two independent equations intersect in a straight line, which can equally be written parametrically as x=3t4, y=7t+2, z=9t. The pattern generalises without change: in n, the solution set of a consistent linear system is a translate of a subspace — a flat object of some dimension between 0 and n. This is why the solution set is never a stray collection of isolated points: once two solutions exist, every point on the line joining them is a solution too.

The two permitted operations, and why they suffice

Addition and scalar multiplication generate every construction in the subject. Linear combination, span, matrix-vector product, matrix multiplication, determinant expansion and change of basis are all built from repeated use of exactly these two operations. Nothing else is needed, and admitting anything else — products of unknowns, powers, logarithms — would destroy the property that makes the subject tractable. The habitual list of forbidden forms is worth memorising: xy, x2, x3/x4, tan(ab) and log(cd) all disqualify an equation.

Superposition: the property that pays for the restriction

If u and v both satisfy a homogeneous linear system, so does αu+βv for any scalars α,β. This is the superposition principle, and it is the engineering payoff of linearity: responses to separate loads, sources or inputs can be computed independently and summed. Structural analysis under combined load cases, circuit analysis with several sources, and frequency-domain signal processing all rest on it. A nonlinear model forfeits superposition and with it the ability to decompose a problem into independently solvable pieces.

Linearity is a modelling choice, not a property of nature

Very few physical relationships are exactly linear. Hooke's law, Ohm's law and Fourier's law are all first-order approximations that hold over a working range. The engineering practice is linearisation: expand about an operating point, discard second-order terms and accept a model that is valid locally. The resulting system is solvable in closed form, its sensitivity is computable, and its failure is predictable — three properties that usually outweigh the loss of fidelity. Knowing the range over which the linearisation holds is part of the model.

Working beyond three dimensions

Intuition from 2 and 3 is a scaffold, not a foundation. Statements proved algebraically hold in n for every n, including cases with no picture. A structural model with 106 degrees of freedom, a recommender system with 104 latent factors and a spectral method with 103 modes are all handled by the same theorems that describe two intersecting lines. Deliberately keeping the treatment algebraic, with geometry as illustration rather than proof, is what makes that scaling possible.

Why the scalars are complex

The coefficients, constants and unknowns are drawn from , the complex numbers. In practice most systems that arise from measurement have real data and are handled entirely within . The reason for admitting from the start is that a real matrix can have complex eigenvalues, and the characteristic polynomial is guaranteed to factor completely only over . Committing to complex scalars at the outset means the existence theorems later in the subject need no exceptions and no restatement.

Decision path: is this equation linear?

Declare the unknownsList which symbols are to be solved for. Every other symbol is data. Linearity is judged only with respect to this list.
Expand every product and functionMultiply out brackets and remove any composite expressions so each term is visible in isolation.
Inspect each termA term is admissible only if it is a constant, or a constant multiplied by exactly one unknown to the first power.
Reject products and powers of unknownsTerms such as x1x2, x12, x1/x2, x1 or tan(x1) disqualify the equation.
Collect into standard formRewrite as a1x1++anxn=b with all unknowns on the left and the constant on the right.
If nonlinear, consider a change of unknownsSubstituting new symbols for nonlinear groupings, or fitting coefficients rather than arguments, often recovers a linear system.

Equations

Standard form of a linear equation

EQ-WILA-01
a1x1+a2x2+a3x3++anxn=b

The defining shape. Each unknown xj appears once, to the first power, multiplied by a constant coefficient aj, and the terms are combined by addition alone.

A system of m linear equations in n unknowns

EQ-WILA-02
a11x1+a12x2++a1nxn=b1a21x1+a22x2++a2nxn=b2am1x1+am2x2++amnxn=bm

The central object of the subject. The double subscript on aij records equation i and unknown j; all data are drawn from .

Flat sets in two and three dimensions

EQ-WILA-03
y=mx+b,ax+by+cz=d

A line in the plane and a plane in space. Both are solution sets of single linear equations; the flatness is exactly the absence of powers and products of the unknowns.

Parametric description of a line in space

EQ-WILA-04
{x=3t4y=7t+2z=9tt

The same flat set described by sweeping a parameter rather than by imposing constraints. Every coordinate is an affine function of t, which is what keeps the object straight.

Forms that are not linear

EQ-WILA-05
xy+5yz=13,x1+x32x4x3x4x52=0,tan(ab)+log(cd)=7

Each fails for a different reason: a product of unknowns, a power and a quotient of unknowns, and unknowns inside transcendental functions. None can be brought to standard form.

Superposition property

EQ-WILA-06
L(αu+βv)=αL(u)+βL(v)

The algebraic statement of linearity for a map L. Scaling and combining inputs scales and combines outputs, which is what allows load cases, sources and modes to be analysed independently.

Variable Definitions

Symbols used on this page
SymbolNameMeaningDomain / type
xjUnknownThe j-th quantity to be solved forcomplex number
aijCoefficientMultiplier of unknown j in equation i; known datacomplex number
biConstant termRight-hand side of equation icomplex number
mEquation countNumber of simultaneous constraints imposedpositive integer
nUnknown countNumber of quantities being solved forpositive integer
αScalarConstant multiplier used in scaling and linear combinationcomplex number
tParameterFree quantity sweeping out a parametrically described flat setreal or complex number
Complex scalarsThe number system from which all coefficients, constants and unknowns are drawnfield
LLinear mapA function respecting addition and scalar multiplicationfunction on vectors

Worked Numerical Example

Problem statement

A thermocouple is calibrated against a reference bath. The response is known to follow a quadratic law y=c0+c1t+c2t2 in the bath temperature t, and three calibration points have been recorded. Show that this manifestly nonlinear relationship yields a linear system in the calibration coefficients, and solve it.

  1. Identify the unknowns

    The bath temperature t and the reading y are both measured, so they are data. The quantities to be determined are the three calibration coefficients c0, c1 and c2. Linearity must be assessed with respect to that list, not with respect to t.

  2. Substitute the measured pairs

    The recorded points are (t,y)=(0,2), (1,3) and (2,8) in working units. Substituting each into the model gives one equation per measurement:

    c0+c1(0)+c2(0)2=2c0+c1(1)+c2(1)2=3c0+c1(2)+c2(2)2=8
  3. Collect into standard form

    Each squared quantity is a number, not an unknown, so it is simply a coefficient. The system is linear in c0,c1,c2 with m=3 and n=3:

    c0+0c1+0c2=2c0+1c1+1c2=3c0+2c1+4c2=8
  4. Eliminate the offset

    The first equation gives c0=2 immediately. Substituting into the remaining two removes c0 and leaves a two-by-two system:

    c1+c2=1,2c1+4c2=6
  5. Solve the reduced system

    Halving the second equation gives c1+2c2=3. Subtracting c1+c2=1 yields c2=2, and back-substitution gives c1=12=1.

    c0=2,c1=1,c2=2
  6. Verify against the original data

    At t=0: 2. At t=1: 21+2=3. At t=2: 22+8=8. All three measurements are reproduced exactly, so the calibration curve is y=2t+2t2.

  7. Note what made this work

    The curve is a parabola — there is nothing straight about it. Linearity entered because the unknowns were the coefficients, which enter the model additively and are scaled only by known powers of t. The same manoeuvre underlies polynomial regression, spline fitting, finite element assembly and the design of finite impulse response filters.

Result

Three measurements determine three calibration coefficients uniquely: y=2t+2t2. The instrument can now be inverted over its working range, and because the fitting problem was linear the sensitivity of each coefficient to measurement error is itself computable by linear methods.

Applications & Industry Use

Structural engineering

Load-case superposition

Elastic structural analysis assumes small displacements, which makes the stiffness relation between load and deflection linear. Dead load, live load, wind and thermal cases are then analysed separately and combined by scaled addition, which is why design codes can specify load factors applied to independently computed results.

Electrical engineering

Circuit analysis with multiple sources

Resistive and reactive networks obeying Ohm's law give linear relations between node voltages and branch currents. Superposition allows each independent source to be considered in isolation with the others suppressed, and the responses added — a technique that is only valid because the network model is linear.

Aerospace & control

Linearisation about a trim condition

Aircraft flight dynamics are strongly nonlinear, yet controllers are designed on a linear model obtained by expanding the equations of motion about a steady trim point. Gain scheduling then blends several such linear models across the flight envelope, each valid over a limited range of perturbation.

Computer graphics

Geometric transformation pipelines

Rotation, scaling, shear, projection and change of viewpoint are all linear or affine maps. Because composition of linear maps is again linear, an entire transformation chain collapses into a single matrix that a graphics processor applies per vertex, which is the basis of real-time rendering performance.

Operations research

Production planning and blending

Resource constraints, recipe balances and capacity limits in production planning are naturally linear in the quantities produced. Linear programming exploits this: the feasible region is a flat-faced polytope, and the optimum is guaranteed to occur at a vertex, which makes large planning problems tractable.

Data science & statistics

Regression and feature models

Ordinary least squares is a linear system in the model coefficients regardless of how the predictors were constructed. Polynomial terms, interaction terms, Fourier features and basis-function expansions all leave the estimation problem linear, which is why regression fitting is a single deterministic computation rather than an iterative search.

Design Considerations

State the operating range of any linearisation

A linearised model is a local approximation. Its validity depends on the perturbation remaining small relative to the curvature of the underlying relationship. Any model handed on to another engineer should carry an explicit statement of the operating point and the range over which the linear approximation was judged acceptable.

Choose the unknowns to preserve linearity

Many apparently nonlinear identification problems become linear under a change of unknowns. Fitting coefficients instead of exponents, working with logarithms of multiplicative models, or introducing an auxiliary variable for a nonlinear grouping can all recover a solvable linear system. The choice is made at the modelling stage and is difficult to revisit later.

Distinguish linear from affine when the equation becomes a function

An equation with a non-zero right-hand side has a solution set that misses the origin, so the associated map does not preserve scaling. As long as the object of interest is the solution set, the distinction is cosmetic. As soon as the same expression is treated as a transformation to be composed or inverted, the constant term must be handled explicitly, typically by adding a homogeneous coordinate.

Scale the variables before computing

Linearity says nothing about numerical conditioning. A system mixing millimetres with megapascals produces coefficients spanning many orders of magnitude and a needlessly ill-conditioned matrix. Non-dimensionalising or equilibrating rows and columns before solving costs almost nothing and often recovers several digits of accuracy.

Count equations against unknowns early

The relationship between m and n sets expectations before any arithmetic is done. Fewer equations than unknowns cannot give a unique answer; more equations than unknowns will usually be inconsistent with measured data and calls for a least-squares formulation rather than an exact solve. Establishing this at the modelling stage prevents wasted effort.

Decide whether complex scalars are required

Static equilibrium, mass balance and resistive networks stay real. Modal analysis, alternating-current impedance and any problem whose answer involves phase or oscillation will produce complex quantities even from real input data. Choosing a real storage type early and discovering complex eigenvalues late is an expensive correction.

Standards & Codes

Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.

Applicable standards, conventions and reference implementations
ReferenceTitleRelevance to this topic
ISO 80000-2Quantities and units — Part 2: MathematicsPrescribes the notation used throughout: italic for scalar variables, upright for operator names, and the conventions for indexed coefficients aij that make a system of equations unambiguous across languages.
ISO 80000-1Quantities and units — Part 1: GeneralGoverns quantity calculus and unit consistency. A linear equation is only meaningful when every term carries the same dimension, which is the first check on any model before it is solved.
IEEE 754-2019IEEE Standard for Floating-Point ArithmeticDefines the arithmetic in which nearly all linear models are actually evaluated, including the rounding behaviour that turns an exactly linear model into an approximately linear computation.
ISO/IEC 40314Mathematical Markup Language (MathML) Version 3.0The markup used to encode the equations on this page so they are machine-readable, searchable and exposed to assistive technology rather than rendered as images.
BLAS Level 1Basic Linear Algebra Subprograms, vector operationsCodifies the two permitted operations directly: xAXPY computes αx+y and xSCAL computes αx. The entire numerical stack is built on these primitives.

Material Selection

For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.

Representation and precision selection
RepresentationSelect whenTrade-off
IEEE 754 binary64 (double precision)Default for engineering computation where input data carries measurement uncertainty and problem sizes are moderate to large.About sixteen significant decimal digits at eight bytes per number; ample for most models but silently loses exactness, so equality tests must become tolerance tests.
IEEE 754 binary32 (single precision)Graphics pipelines, embedded targets and accelerator workloads where memory bandwidth dominates and the model is well conditioned.Halves storage and often doubles throughput, but roughly seven significant digits leaves little margin once a moderately conditioned system is solved.
Exact rational arithmeticSmall symbolic models, textbook verification and any situation where the structural answer must be certified rather than estimated.No rounding at all, but operand size grows during elimination and performance degrades sharply with problem size.
Real scalars onlyStatics, mass and energy balances, resistive networks and geometric transformations, where no oscillatory or spectral quantity appears.Halves memory and simplifies code, but forecloses eigenvalue and frequency-domain analysis without a data-type change later in the project.
Complex scalarsAlternating-current analysis, modal and vibration studies, signal processing and any spectral method.Doubles storage and roughly quadruples multiplication cost, in exchange for a theory with no missing cases and no special handling of oscillatory modes.
Fixed-point arithmeticDeeply embedded controllers and digital signal processors without floating-point hardware.Deterministic timing and minimal silicon area, at the cost of manual scaling analysis and a hard overflow boundary that must be proved unreachable.

Manufacturing Notes

Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.

Recognising linearity mechanically

Expand every bracket and composite expression, then examine each resulting term in isolation. A term survives only if it is a constant or a constant times a single unknown to the first power. This test is purely syntactic once the unknowns have been declared, which is why symbolic algebra systems can classify an equation without any understanding of the underlying model.

Assembling a system from a model

The standard workflow is: declare the unknowns and fix their order; write one equation per physical constraint; expand and collect terms so that unknowns appear on the left in the fixed order and constants on the right; record any absent unknown as a coefficient of zero rather than omitting it. Preserving zero coefficients explicitly is what makes the coefficient array rectangular and machine-processable.

Library entry points

In NumPy a system is assembled as a two-dimensional array and solved with numpy.linalg.solve; SciPy adds banded, sparse and least-squares variants; SymPy provides exact solution via linsolve for symbolic work. All of these expect the coefficient array in the fixed unknown ordering established at assembly time, so that ordering should be recorded alongside the data.

Cost of the basic operations

Scaling a vector of length n costs n multiplications; adding two costs n additions; forming a general linear combination of k vectors costs about 2kn operations. Every higher-level operation in the subject is assembled from these, which is why operation counts throughout linear algebra are polynomial in the dimensions and predictable in advance.

Verifying a linear model against data

Linearity in the unknowns can be tested empirically: perturb one unknown, record the change in each equation residual, and confirm the change is proportional to the perturbation and independent of the other unknowns. A residual that varies with perturbation size, or that depends on the level of a second unknown, exposes a nonlinearity that the model was assumed not to have.

Failure Modes & Common Mistakes

Failure modes, root causes and prevention
Failure mode / mistakeImpactRoot causePrevention & detection
Declaring the wrong unknownshighTreating a data symbol as unknown, or vice versa, so an equation that is linear in the intended sense is classified as nonlinear.Write the list of unknowns explicitly before assessing any equation, and keep it fixed for the duration of the analysis.
Products of unknowns hidden inside bracketshighAn expression such as x1(x2+3) looks admissible until expanded and reveals the term x1x2.Expand every product fully before classifying, and never assess linearity from a factored form.
Applying superposition to a nonlinear modelhighAdding independently computed responses in a system with material nonlinearity, large displacements or saturation, where superposition does not hold.Confirm linearity of the governing relations over the full range of the combined case before summing separately computed results.
Extrapolating a linearised modelhighUsing a model valid near an operating point at conditions far outside the range over which the linearisation was derived.Record the operating point and the validity range with the model, and enforce a range check at every point of use.
Dimensional inconsistency between termsmediumTerms with mismatched units are added because the algebra is manipulated without carrying dimensions through.Check that every term in every equation reduces to the same dimension before solving; a dimensional error is nearly always an algebraic error.
Confusing linear with affine when composing mapsmediumTreating an equation with a non-zero constant term as a scaling-preserving map, so composition or inversion drops the translation.Represent affine maps with an explicit constant vector, or use homogeneous coordinates so composition stays a single matrix product.
Assuming intuition from two dimensions transfersmediumReasoning about n from pictures of intersecting lines, which misleads about independence, orthogonality and volume in high dimension.Use geometry to illustrate and algebra to prove; verify any dimensional claim against the algebraic statement.
Choosing real storage for a problem with complex answerslowCommitting to real arithmetic in code before discovering that the model produces oscillatory modes or complex eigenvalues.Assess at the modelling stage whether spectral or frequency-domain quantities will be required, and select the scalar type accordingly.

FAQs

Why is y=mx+b called linear when it does not pass through the origin?

The label reflects the geometry of the solution set, which is a straight line, rather than the algebraic property of preserving scaling. Strictly the relation is affine: doubling x does not double y unless b=0. The convention is universal and harmless when the object of interest is the solution set, but the distinction becomes material once the same expression is treated as a transformation to be composed or inverted.

Can an equation be linear in one variable and nonlinear in another?

Yes, and this is one of the most useful facts in applied mathematics. The relation y=c0+c1t+c2t2 is quadratic in t and perfectly linear in c0,c1,c2. Fitting the coefficients to measured data is therefore a linear problem, which is why polynomial regression, spline fitting and finite element assembly all reduce to solving linear systems.

Is division ever allowed in a linear equation?

Division by a non-zero constant is allowed, because it is multiplication by the reciprocal of that constant. Division by an unknown is never allowed, since it produces a term such as x1/x2 that cannot be written as a constant times a single unknown. The same rule distinguishes a permitted 2x from a forbidden x.

What does linear algebra actually buy an engineer over calculus?

Completeness and determinism. For a linear system there is a finite procedure that always terminates, always characterises the full solution set, and reports unambiguously whether that set is empty, a single point or an infinite family. No comparable guarantee exists for general nonlinear systems, where solvers may fail to converge, find one root among many, or give no information about what was missed.

If most real systems are nonlinear, why is linear algebra so dominant?

Because nonlinear methods are built from linear ones. Newton's method solves a linear system at every iteration, finite element analysis assembles and solves a linear system at every load step, and optimisation algorithms use linear models of the objective at each point. Improving the linear kernel improves every method layered on top of it.

Why work over the complex numbers when engineering data is real?

Because real matrices routinely have complex eigenvalues, and the characteristic polynomial factors completely only over . Admitting complex scalars from the outset means the existence theorems later in the subject need no exceptions. In practice, computation with real data stays real until a spectral question is asked.

How many equations do I need to determine n unknowns?

At least n independent ones. Fewer than n independent equations always leaves an infinite solution set; exactly n independent equations give a unique solution; more than n equations are usually inconsistent with measured data and should be handled as a least-squares problem rather than an exact solve. Independence, not raw count, is the operative condition.

References

  1. Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section WILA. Licensed under the GNU Free Documentation License v1.2.
  2. ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
  3. Strang, G. Introduction to Linear Algebra, 5th edition. Wellesley-Cambridge Press, 2016.
  4. IEEE 754-2019, IEEE Standard for Floating-Point Arithmetic. Institute of Electrical and Electronics Engineers.
  5. Lawson, C. L., Hanson, R. J., Kincaid, D. R. and Krogh, F. T. Basic Linear Algebra Subprograms for Fortran Usage. ACM Transactions on Mathematical Software, 1979.

AI Suggested Questions

  • Take the nonlinear relation y=Aekt and show me every change of variables that turns parameter estimation into a linear system, with the cost of each.
  • Derive the linearised equations of motion for a pendulum about the inverted equilibrium and state the angular range over which the approximation holds to one per cent.
  • Explain precisely where superposition fails in a structure with geometric nonlinearity, and what a load-case summation would get wrong.
  • Compare the number of floating-point operations in solving a 1000-unknown linear system against one Newton iteration on the equivalent nonlinear system.
  • Give me three engineering models that look linear but are not, and show the hidden product or power of unknowns in each.
  • Why does the fundamental theorem of algebra force the scalars in linear algebra to be complex rather than real?

Related Calculators

Continue learning

Algebraic and Geometric Multiplicities of Eigenvalues | KEVOS® MathematicsArticle · Project ManagementAmitsur’s Theorem on the Radical of a Polynomial Ring | KEVOS®Article · Project ManagementAmitsur’s Theorem on the Radical of an Algebra of Small Dimension | KEVOS®Article · Project ManagementArchetypes: Reference Catalogue of Worked Systems | KEVOS® MathematicsArticle · Project Management