Engineering/Mathematics/Systems of Linear Equations
Matrix and Vector Notation for Linear Systems
A system of linear equations written out in full is unreadable beyond three or four unknowns. Replacing it with a coefficient matrix, a vector of constants and a solution vector turns pages of subscripted arithmetic into a single expression that the rest of linear algebra can operate on.
- Core level
- Stream: matrix-algebra
- Reading time 12 min
- Ref KVS-ENG-MATH-0013
- Taxonomy
- Engineering / Mathematics
- Prerequisite
- Systems of linear equations; augmented matrices
- Compact form
- for
- Coefficient matrix
- , entries
- Homogeneous case
- Entry notation
- is a single scalar
Overview
Writing a system of linear equations in full requires one line per equation, each carrying coefficients, variable names, plus signs and a constant. For a system of any realistic size the notation dominates the content. The remedy is to separate the three distinct objects that the written form conflates: the array of coefficients, the list of constants, and the list of unknowns. Each becomes a named object in its own right, and the system becomes a relation between them.
The building block is the column vector: an ordered list of scalars written vertically, whose -th entry is denoted . That bracket notation deserves care, because it looks substantial but denotes a single number. Two column vectors of size can be compared entry by entry, added, or scaled; the collection of all such vectors is written . The zero vector, every entry of which is zero, plays a distinguished role because it is the right-hand side of every homogeneous system.
From a system with equations and unknowns three objects are extracted. The coefficient matrix is the array whose entry is the coefficient of in equation . The vector of constants collects the right-hand sides into a column of size . The solution vector collects the unknowns into a column of size . Sizes are not interchangeable: lives in and in , and confusing them is one of the most persistent sources of error in setting up a model.
With those names in place the whole system is written , and its augmented matrix as . The payoff is not brevity for its own sake. It is that the coefficient matrix can now be studied independently of any particular right-hand side, which is precisely what makes the null space, the column space, rank and nonsingularity available as concepts. Every later structural result in the subject is a statement about made possible by this separation.
Definition
Column Vector
CVA column vector of size is an ordered list of scalars written vertically from top to bottom. The entry in position of the vector is written . Vectors are named with lower-case letters from the end of the Latin alphabet — , , , , , — and the set of all column vectors of size with complex entries is written .
The expression denotes a single scalar, not a vector. Its visual weight is misleading; treat it exactly as one would treat a subscripted number.
Zero Vector
ZVThe zero vector of size , written , is the column vector every entry of which is the number zero: for . Its size is inferred from context and is not carried in the symbol, so care is needed when the same expression involves zero vectors of two different sizes.
Coefficient Matrix
CMFor a system of linear equations in unknowns, the coefficient matrix is the matrix whose entry is the coefficient of in equation . A variable missing from an equation contributes an entry of zero; every position in the array must be filled.
The coefficient matrix is a property of the left-hand sides alone. Two systems with identical left-hand sides and different constants share the same , which is exactly why can be studied on its own.
Vector of Constants and Solution Vector
VOCThe vector of constants is the column vector of size whose -th entry is the constant term of equation . The solution vector is the column vector of size whose -th entry is the unknown . The symbol does double duty: it names the list of variable quantities, and it names a particular assignment of values that satisfies the system.
The sizes differ whenever . Writing where belongs is a size error that a dimensional check catches immediately.
Concepts
The system in entry notation
Equation of the system states . Read across the row index from to and this single expression reproduces every equation. The compressed form makes the structure visible: the row index of selects an equation and the column index selects an unknown. Once that correspondence is fixed, every operation on the system — scaling an equation, adding one equation to another, eliminating a variable — becomes an operation on rows of with the same operation applied to .
The shorthand and what it commits you to
Writing names a system by its two data objects. The notation carries a genuine claim: the system is completely determined by the pair, so any two systems with the same coefficient matrix and the same vector of constants are the same system, whatever names the variables were given. It also makes the homogeneous companion expressible with no extra machinery — is the system obtained by keeping the coefficients and discarding the loading.
The augmented matrix as a single computational object
The augmented matrix places the constants as an extra column to the right of the coefficients, separated by a vertical rule that carries no arithmetic meaning but marks the boundary between the two roles. Row operations are then applied to a single array rather than to a matrix and a vector in parallel. This is the form in which reduction algorithms actually operate, and the separation rule is what allows the pivot test on column to be stated.
Why the zero constants column of a homogeneous system can be dropped
Every elementary row operation maps zero to zero: scaling preserves it, swapping permutes zeros, and adding a multiple of a zero entry leaves a zero. The final column of therefore remains zero throughout the reduction and carries no information. It is standard practice to reduce alone for a homogeneous system, which is also why the resulting theory can be phrased as a property of the matrix rather than of a system.
Vector equality as a compression device
Two column vectors of the same size are equal precisely when they agree in every entry. That definition is what permits a whole system to be written as a single equality between two vectors: the left-hand side is a column whose -th entry is the linear expression from equation , and the right-hand side is . One vector equation of size is exactly scalar equations, so nothing has been lost. This is the mechanism by which the language of systems of equations is gradually replaced by the language of vectors.
Size discipline
The three objects have three different shapes: is , has size and has size . Almost every notational error in setting up a linear model is a size error, and almost all of them are caught by asking whether the shapes compose. This discipline also explains why the augmented matrix has columns rather than , and why the index appears so often in consistency arguments.
Procedure: converting a written system into matrix and vector form
Equations
Column vector and its entries
EQ-MVN-01An ordered vertical list of scalars. The bracket notation extracts a single entry and yields a scalar, not a vector of size one.
Coefficient matrix
EQ-MVN-02Row holds the coefficients of equation ; column holds the coefficients of unknown across all equations.
Vector of constants and solution vector
EQ-MVN-03The constants are indexed by equation and the unknowns by variable, so the two vectors have different sizes whenever the system is not square.
The system in entry form
EQ-MVN-04A single expression that reproduces every equation of the system as the row index runs over its range.
Compact and augmented notation
EQ-MVN-05Three equivalent presentations of the same system: as a named pair, as a matrix equation, and as the single array on which row reduction operates.
Zero vector
EQ-MVN-06The vector every entry of which is zero. Its size is carried by context, so an expression may legitimately contain zero vectors of two different sizes.
Homogeneous system in compact form
EQ-MVN-07Because the constants column of a homogeneous system remains zero under every row operation, the augmented matrix may be replaced by the coefficient matrix alone with no loss.
Variable Definitions
| Symbol | Name | Meaning | Domain / type |
|---|---|---|---|
| Coefficient matrix | Array of the coefficients of the unknowns, one row per equation | m x n matrix over C | |
| Matrix entry | The coefficient of in equation ; a single scalar | element of C | |
| Vector of constants | Right-hand sides of the equations, in equation order | vector in C^m | |
| Solution vector | The unknowns, or a particular assignment satisfying the system | vector in C^n | |
| Zero vector | Column vector with every entry zero; the right-hand side of a homogeneous system | vector in C^m | |
| Vector entry | The -th component of the column vector ; a scalar | element of C | |
| Equation count | Number of equations, equal to the number of rows of and the size of | positive integer | |
| Unknown count | Number of variables, equal to the number of columns of and the size of | positive integer | |
| Space of column vectors | The set of all column vectors of size with complex entries | set |
Worked Numerical Example
Problem statement
A three-stream blending operation is described by three balance equations in three unknown stream rates. Convert the written system into matrix and vector form, name it, solve it, and state its homogeneous companion.
Write the system with a fixed variable order
The balances are , and . The variable order is fixed now and will define the column order of the coefficient matrix.
Extract the coefficient matrix
Row of holds the coefficients of equation in the fixed variable order. Here , so is square.
Extract the vector of constants and name the unknowns
The constants are collected in equation order into a column of size , and the unknowns into a column of size . The system is then referenced as .
Form the augmented matrix
For computation the two objects are placed side by side in one array. The vertical rule marks the boundary between coefficients and constants and carries no arithmetic meaning.
Reduce and read the solution
Row reduction gives the identity matrix in the coefficient block, so and the solution is unique. It is read directly from the final column.
Verify by substitution
Check against the original equations, not the reduced ones: ; ; . All three hold, so is the solution vector in the second sense — an assignment of values, not merely a list of names.
State the homogeneous companion
Replacing by while keeping gives . Because the coefficient block already reduces to the identity, and the only solution is the trivial one. The companion system therefore certifies that is nonsingular — a statement about the matrix that no particular right-hand side could have provided.
The written system, the pair and the augmented array are three presentations of one object, and the solution is the unique feasible set of stream rates. Separating the coefficients from the constants also yields a result about the process independent of this particular demand: because the homogeneous companion has only the trivial solution, any demand vector whatsoever produces exactly one feasible operating point.
Applications & Industry Use
Mass balance models in matrix form
Flowsheet balances are assembled directly into a coefficient matrix whose columns are streams and whose rows are components. Storing the model as rather than as written equations allows the demand vector to be varied across scenarios while the process structure, encoded once in , remains untouched.
Array layout and library interfaces
Numerical libraries accept exactly this decomposition: a two-dimensional array for and a one-dimensional array for . Understanding that has size while has size is what prevents the most common interface error, which is passing arrays of mismatched length to a solver.
Stiffness assembly
Finite element assembly builds a global coefficient matrix from element contributions and a separate load vector. The separation is essential to the workflow: the stiffness matrix is factorised once and reused across many load cases, which is only meaningful because is independent of .
Nodal admittance formulation
Circuit analysis writes the network as an admittance matrix times a vector of node voltages equal to a vector of injected currents. Naming the three objects separately allows the same admittance matrix to be reused for every excitation, and the source-free case to be studied as the homogeneous companion.
Observation equations
Each measurement contributes one row to a design matrix and one entry to an observation vector. Because there are usually far more observations than unknown coordinates, keeping and notationally distinct is what makes the over-determined structure of the problem explicit before any adjustment is attempted.
Input-output and resource models
Technology coefficients form a matrix, final demand forms a vector, and activity levels form the unknown vector. The compact notation makes it natural to ask which demands are attainable — a question about the columns of the matrix rather than about any single scenario.
Design Considerations
Fix and document the variable ordering
The correspondence between column index and unknown is a convention that must be recorded, because nothing in the matrix itself carries the variable names. Models that are exchanged between teams should ship an explicit column dictionary; permuting columns without permuting the interpretation is a silent and serious error.
Enter zero coefficients explicitly
A variable absent from an equation contributes a coefficient of zero, and the position must be filled. Omitting it shifts every subsequent coefficient left by one and produces a syntactically valid but wholly incorrect matrix. When transcribing by hand, count entries per row against before proceeding.
Keep separate from in storage
Store the coefficient matrix and the constants as distinct objects and form the augmented array only when a reduction requires it. This preserves the ability to factorise once and solve for many right-hand sides, which is the single largest practical saving available in linear system work.
Check shapes before checking numbers
A dimensional inconsistency between , and is cheaper to detect than an arithmetic error and more likely to be present. Any assembly routine should assert that the row count of equals the size of and the column count equals the size of before the first floating-point operation.
Respect the vertical rule as documentation only
The bar in has no arithmetic effect; row operations cross it freely. Its purpose is to record which column is the constants column, and hence which column the consistency test inspects. Losing the rule during transcription is what turns an augmented matrix into an ordinary one and invites the constants to be treated as another variable.
Do not overload the symbol carelessly
The solution vector names both the list of unknowns and a particular numeric solution. The ambiguity is standard and usually harmless, but in written work it is worth distinguishing the two — for instance by reserving a separate symbol for a specific solution when several are under discussion.
Standards & Codes
Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.
| Reference | Title | Relevance to this topic |
|---|---|---|
ISO 80000-2 | Quantities and units — Part 2: Mathematics | Prescribes the notation used here: italic letters for scalar variables, bold or arrow notation for vectors, bracketed arrays for matrices, and the conventions for indexing entries such as . |
ISO/IEC 14882 | Programming languages — C++ | Defines std::valarray and the container semantics that determine how a coefficient matrix and a vector of constants are laid out in memory, including the row-major storage that matches the index convention used here. |
BLAS Level 2 | Basic Linear Algebra Subprograms, matrix-vector operations | Standardises the xGEMV interface for computing , whose argument list is exactly the decomposition into matrix, vector and dimensions that this notation makes explicit. |
ISO/IEC 40314 | Mathematical Markup Language (MathML) Version 3.0 | Provides the mtable structures that encode column vectors and augmented arrays semantically, so that the vertical partition of survives as structure rather than as a drawn line. |
W3C WCAG 2.1 AA | Web Content Accessibility Guidelines | Requires that matrices and vectors be presented as semantic markup with alternative text rather than as images, so that the entry structure of and is available to assistive technology. |
Material Selection
For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.
| Representation | Select when | Trade-off |
|---|---|---|
| Dense two-dimensional array, row-major | Small to medium systems where every coefficient is likely to be non-zero. | Simplest mapping from the written system to storage and the fastest for row operations, but memory grows as regardless of how many entries are zero. |
| Compressed sparse row (CSR) | Network, circuit and finite-element models where most coefficients are zero. | Memory proportional to the number of non-zeros and fast matrix-vector products, at the cost of expensive structural modification and awkward hand inspection. |
| Separate matrix and right-hand-side storage | Many right-hand sides are to be solved against one coefficient matrix. | Enables a single factorisation to be reused, but requires the calling code to maintain the size relationship explicitly rather than relying on a single augmented array. |
| Fused augmented array | A one-off reduction where the reduced form itself is the deliverable. | Keeps the reduction simple and guarantees the constants are carried, but forgoes reuse and blurs the distinction between coefficients and data. |
| IEEE 754 binary64 entries | General engineering computation with measured or derived coefficients. | Adequate range and precision for most models, but exact structural zeros must be recorded as such rather than allowed to arise from cancellation. |
| Exact rational or integer entries | Stoichiometric, combinatorial and small symbolic models where coefficients are exact by construction. | Preserves the exact structure of and makes rank statements defensible, but is unavailable in most numerical toolchains and scales poorly. |
Manufacturing Notes
Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.
Assembly cost and layout
Building and from a written model is work and is usually dominated by input handling rather than arithmetic. Storing row-major matches the row operations that follow, giving contiguous access during elimination; a column-major layout, as used by Fortran and LAPACK, favours column operations and matrix-vector products instead.
Hand transcription discipline
Write the variable order at the head of the columns, then fill row by row, entering an explicit zero for every absent variable. Count the entries of each completed row against before moving on. Most transcription errors are omissions of a zero, and they are far cheaper to catch at this stage than after a reduction.
Library conventions
NumPy represents as a two-dimensional ndarray of shape (m, n) and as a one-dimensional array of length m, with A @ x implementing the matrix-vector product. MATLAB and Octave follow the same decomposition with the backslash operator. The augmented array is formed explicitly with a horizontal stack only when the reduced form itself is wanted.
Verification of an assembled model
Two cheap checks catch most assembly faults. Confirm the shapes, then evaluate for a vector whose entries are known to reproduce a hand-computed right-hand side — a vector of ones is often enough, since it sums each row. Row sums that disagree with the hand computation localise the fault to a specific equation.
Scaling and equilibration
Rows of derived from equations expressed in different physical units can differ by many orders of magnitude, which degrades the conditioning of any subsequent reduction. Scaling each equation so that its largest coefficient has magnitude near one, and applying the same scaling to the corresponding entry of , is a standard preprocessing step that costs and preserves the solution set exactly.
Failure Modes & Common Mistakes
| Failure mode / mistake | Impact | Root cause | Prevention & detection |
|---|---|---|---|
| Omitting a zero coefficient | high | A variable absent from an equation is skipped rather than entered as zero, shifting all later coefficients one column left. | Count the entries of every row against during transcription, and assemble from a fixed variable list rather than by reading each equation independently. |
| Confusing the sizes of and | high | Assuming both vectors have the same length, which holds only when . | Assert that the size of equals the row count of and the size of equals its column count before any computation. |
| Losing the augmented column boundary | medium | Transcribing without recording which column holds the constants. | Record explicitly alongside the array, so that the consistency test knows which column to inspect. |
| Permuting columns without permuting the interpretation | high | Reordering variables for convenience while leaving the variable dictionary unchanged. | Treat the column ordering as part of the model definition and version it with the matrix; column operations are not row operations and change the meaning of the result. |
| Treating as a vector | medium | The bracket notation looks like a vector expression but denotes a single scalar entry. | Read as a number in every context, and check dimensionality whenever it appears inside another vector expression. |
| Ambiguous zero vector size | low | Using for vectors of two different sizes in the same expression without stating which is which. | State the size in words at first use, particularly in identities such as where the two occurrences have sizes and . |
| Failing to normalise equations before assembly | medium | Variables left on the right-hand side or constants left on the left, producing sign errors throughout a row. | Rearrange every equation into the standard form before extracting a single coefficient. |
| Mixing row-major and column-major conventions | medium | Passing a row-major array to an interface expecting column-major storage, silently transposing the matrix. | Check the storage convention of every library boundary, and validate with a deliberately non-symmetric test matrix. |
FAQs
Why write rather than ?
The two say the same thing, but the pair notation emphasises that a system is determined by its coefficient matrix and its constants alone, with the unknown playing no role in the specification. It also names the object without asserting anything about solvability, which is useful when the very question at hand is whether a solution exists.
Does the vertical bar in the augmented matrix do anything arithmetically?
No. Row operations act across the whole row, bar included. The rule is documentation: it records which column holds the constants, and therefore which column the consistency test examines. Some texts omit it entirely and simply state the value of .
Why are vectors written as columns rather than rows?
So that the matrix-vector product and the interpretation of a matrix's columns compose correctly. With a column, is a linear combination of the columns of with the entries of as coefficients — a reading that underpins the column space and the whole treatment of spanning sets.
What is the difference between and ?
is an ordered list of scalars; is one of those scalars. The bracket notation is visually heavy, which misleads readers into treating it as a compound object. In any dimensional check it must be counted as a single number.
Can I drop the constants column when reducing?
Only for a homogeneous system, where the column is zero and stays zero under every row operation. For any other system the constants carry the information that decides consistency, and discarding them makes the reduction unable to detect an inconsistent system at all.
Does the notation assume real numbers?
No. The scalars are drawn from the complex numbers, and denotes complex column vectors. Restricting to real entries gives , and every definition on this page reads identically. The complex setting is chosen because it makes later eigenvalue results uniform.
How does this notation help once the system is solved?
It separates what is being asked from what the model is. Because is defined without reference to , questions such as which right-hand sides are solvable, how many solutions there are, and whether the model is well posed become questions about the matrix alone — the null space, the column space and the rank.
References
- Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section HSE, subsection MVNSE. Licensed under the GNU Free Documentation License v1.2.
- ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
- Dongarra, J. J. et al. 'An extended set of FORTRAN basic linear algebra subprograms'. ACM Transactions on Mathematical Software, 14(1), 1988.
- Anderson, E. et al. LAPACK Users' Guide, 3rd edition. Society for Industrial and Applied Mathematics, 1999.
- W3C. Mathematical Markup Language (MathML) Version 3.0, 2nd edition (ISO/IEC 40314).
AI Suggested Questions
- Given a system written with variables on both sides of the equals sign, show the normalisation steps needed before the coefficient matrix can be extracted.
- Why does writing vectors as columns rather than rows make the matrix-vector product a combination of the columns of the matrix?
- What goes wrong numerically when the rows of a coefficient matrix differ by many orders of magnitude, and how does equilibration fix it?
- Compare row-major and column-major storage for a coefficient matrix in terms of cache behaviour during Gaussian elimination.
- Show how the same coefficient matrix can be reused across many right-hand sides via a single factorisation, and quantify the saving.
- How would the notation on this page change if the scalars were drawn from a finite field instead of the complex numbers?
Related Calculators
Parse a written system of equations and emit the coefficient matrix, vector of constants and augmented array with shape checks.
Linear System SolverSolve and report the solution vector together with rank and consistency.
Matrix-Vector Product CalculatorEvaluate entry by entry with dimensional validation, useful for verifying an assembled model.
