The Matrix-Vector Product | KEVOS® Mathematics

Q: Why is a matrix-vector product slow relative to its operation count?

Because it reads $mn$ matrix entries to perform only $2mn$ operations, and each entry is used exactly once. The kernel therefore runs at memory bandwidth rather than at arithmetic peak. Batching several vectors into a matrix-matrix product reuses each matrix entry many times and moves the workload into a compute-bound regime, which is why batching is the standard optimisation.

Overview

Linear combinations of the columns of a matrix appear throughout the theory of linear systems: a vector of constants is reachable exactly when it is such a combination, and the coefficients that achieve it are precisely a solution. The matrix-vector product packages that recurring construction into a single operation. For an $m \times n$ matrix $A$ with columns $A_1, A_2, \ldots, A_n$ and a vector $\vec{u}$ of size $n$ , the product $A\vec{u}$ is defined to be $[\vec{u}]_1 A_1 + [\vec{u}]_2 A_2 + \cdots + [\vec{u}]_n A_n$ .

Defining the product this way — the column picture — rather than as a list of row dot products is a deliberate choice. The row formulation is the better computational recipe and is derived later, but the column formulation is what connects the operation to everything already established. It makes immediately visible that the reachable set of $A\vec{x}$ is the span of the columns, that a system is consistent exactly when $\vec{b}$ lies in that span, and that the null space is the set of coefficient vectors producing a cancelling combination.

The conformability rule is dictated by the definition. There must be one coefficient per column, so $\vec{u}$ must have size $n$ ; and each column has $m$ entries, so the result has size $m$ . A rectangular matrix therefore changes the size of the vector it acts on, which is the algebraic statement that a linear map between spaces of different dimension is being applied.

The immediate payoff is notational and conceptual. A system of $m$ equations in $n$ unknowns, previously written as an augmented matrix or as a list of equations, becomes the single equation $A\vec{x} = \vec{b}$ : the solutions of the system and the solutions of the vector equation are the same set. Every subsequent development — column space, matrix multiplication, linear transformations, eigenvectors — is expressed in terms of this product.

Definition

Matrix-Vector Product

MVP

Let $A$ be an $m \times n$ matrix with columns $A_1, A_2, A_3, \ldots, A_n$ , and let $\vec{u}$ be a column vector of size $n$ . The matrix-vector product of $A$ with $\vec{u}$ is the column vector of size $m$ given by the linear combination $A\vec{u} = [\vec{u}]_1 A_1 + [\vec{u}]_2 A_2 + \cdots + [\vec{u}]_n A_n$ .

The product is written by juxtaposition, with the matrix on the left and the vector on the right. The reversed order $\vec{u}A$ is a different and generally undefined expression.

Systems of Linear Equations as Matrix Multiplication

SLEMM

For an $m \times n$ coefficient matrix $A$ and a vector of constants $\vec{b}$ of size $m$ , a vector $\vec{x}$ of size $n$ solves the linear system with coefficient matrix $A$ and constants $\vec{b}$ if and only if it satisfies the vector equation $A\vec{x} = \vec{b}$ . The two solution sets are identical, not merely related.

The proof is a chain of equivalences: $\vec{x}$ solves the system exactly when the columns of $A$ combine with coefficients $[\vec{x}]_j$ to give $\vec{b}$ , which is exactly the statement $A\vec{x} = \vec{b}$ .

Equal Matrices and Matrix-Vector Products

EMMVP

Suppose $A$ and $B$ are $m \times n$ matrices such that $A\vec{x} = B\vec{x}$ for every $\vec{x} \in \mathbb{C}^n$ . Then $A = B$ . Choosing $\vec{x}$ to be the standard unit vector with a single $1$ in position $i$ extracts column $i$ of each matrix, so the columns agree one by one and the matrices are equal.

The hypothesis can be weakened to agreement on the $n$ standard unit vectors alone, or on any basis of $\mathbb{C}^n$ . A matrix is therefore completely determined by its action on a basis.

Concepts

The column picture and what it reveals

Reading $A\vec{u}$ as a weighted sum of columns makes the reachable set of the operation transparent. As $\vec{u}$ ranges over all of $\mathbb{C}^n$ , the products $A\vec{u}$ range over exactly the span of the columns of $A$ — the column space $\mathcal{C}(A)$ . Consistency of $A\vec{x} = \vec{b}$ is therefore equivalent to the membership question $\vec{b} \in \mathcal{C}(A)$ , and the number of independent columns bounds how much of $\mathbb{C}^m$ can be reached. None of this is visible from a row-by-row description.

The row picture and the connection to inner products

The same product can be computed one output entry at a time: $[A\vec{u}]_i = \sum_{k=1}^{n} [A]_{ik}[\vec{u}]_k$ , the sum of products along row $i$ of $A$ against $\vec{u}$ . This is the recipe used in practice, and it identifies each output entry as a weighted sum of the inputs. Geometrically each row defines a hyperplane and the system asks for a point on all of them at once, which is the complementary reading to the column picture. Both descriptions are correct; the column picture explains structure and the row picture explains computation.

Linearity of the product in the vector

Because the product is a linear combination, it inherits linearity in its second argument: $A(\vec{u} + \vec{v}) = A\vec{u} + A\vec{v}$ and $A(\alpha\vec{u}) = \alpha(A\vec{u})$ . This is the algebraic form of superposition and is the reason linear models are tractable at all. Two load cases can be analysed separately and added; a doubled input produces a doubled response; a small perturbation of the input produces a proportional perturbation of the output.

Standard unit vectors extract columns

Let $\vec{e}_i$ be the vector with $1$ in position $i$ and zeros elsewhere. Then $A\vec{e}_i = A_i$ , the $i$ -th column of $A$ , since every other coefficient in the linear combination is zero. This is the mechanism behind the theorem that equal products imply equal matrices, and it is also the practical recipe for recovering a matrix from a black-box linear system: apply the system to each unit input in turn and assemble the responses as columns.

The null space in product form

The null space $\mathcal{N}(A)$ is the set of vectors satisfying $A\vec{x} = \vec{0}$ , which in the column reading is the set of coefficient vectors producing a linear combination of the columns that cancels exactly. A non-trivial null space is therefore precisely a linear dependence among the columns. This reformulation is what links the solution structure of a homogeneous system to the independence of the columns, with no further machinery required.

A matrix is determined by its action

The result that $A\vec{x} = B\vec{x}$ for all $\vec{x}$ forces $A = B$ says a matrix is nothing more than the linear map it induces: two matrices with identical behaviour are identical objects. This licenses proving matrix identities by showing both sides act the same way on an arbitrary vector, which is frequently much easier than an entrywise argument, and it underlies the idea of a matrix representation of a linear transformation.

Decision path: evaluating and interpreting $A\vec{u}$

Check conformability

A

must be

m \times n

and

\vec{u}

must have exactly

n

entries — one coefficient per column. If not, the product is undefined.

Choose a computation routeBy columns, scale each column of

A

by the corresponding entry of

\vec{u}

and add. By rows, form the weighted sum along each row. The two give identical results.

Confirm the result sizeThe output has

m

entries, one per row of

A

. A rectangular matrix changes the size of the vector, which is the point of the operation.

Interpret as a systemIf the target is a specified

\vec{b}

, solving

A\vec{x} = \vec{b}

is the same problem as solving the corresponding linear system; consistency means

\vec{b}

lies in the span of the columns.

Interpret structurallyThe reachable outputs form the column space

\mathcal{C}(A)

; the inputs mapped to

\vec{0}

form the null space

\mathcal{N}(A)

and correspond to dependencies among the columns.

Equations

Matrix-vector product as a linear combination of columns

EQ-MVP-01

Matrix-vector product as a linear combination of columns

The defining formula. Each entry of $\vec{u}$ weights the corresponding column of $A$ ; the sum is a vector of size $m$ .

Entry formula (row picture)

EQ-MVP-02

Entry formula (row picture)

The equivalent computational recipe: output entry $i$ is the weighted sum along row $i$ . This is what implementations evaluate.

A linear system as a vector equation

EQ-MVP-03

A \vec{x} = \vec{b}

The entire system of $m$ equations in $n$ unknowns. Its solution set is identical to that of the original equations, not merely equivalent to it.

Linearity in the vector argument

EQ-MVP-04

A (\vec{u} + \vec{v}) = A \vec{u} + A \vec{v}, A (α \vec{u}) = α (A \vec{u})

Superposition and homogeneity. These two identities are what make a linear model decomposable into independently analysable cases.

Standard unit vectors extract columns

EQ-MVP-05

A {\vec{e}}_{i} = A_{i}, [{\vec{e}}_{i}]_{k} = {\begin{cases} 1 & k = i \\ 0 & k \neq i \end{cases}

Applying the matrix to the $i$ -th unit vector returns the $i$ -th column, which is how a matrix is recovered from a black-box linear system.

Column space and null space in product form

EQ-MVP-06

C (A) = {A \vec{x} ∣ \vec{x} \in ℂ^{n}}, N (A) = {\vec{x} \in ℂ^{n} ∣ A \vec{x} = \vec{0}}

The reachable outputs and the annihilated inputs. Consistency of $A\vec{x} = \vec{b}$ is exactly the condition $\vec{b} \in \mathcal{C}(A)$ .

Determination of a matrix by its action

EQ-MVP-07

A \vec{x} = B \vec{x} \forall \vec{x} \in ℂ^{n} \Rightarrow A = B

Agreement on all vectors — indeed on any basis — forces equality of the matrices, since the unit vectors extract the columns one at a time.

Variable Definitions

Symbols used on this page
Symbol	Name	Meaning	Domain / type
$A$	Matrix	The $m \times n$ matrix acting on the vector	M_{mn}
$A_{j}$	Column of $A$	The $j$ -th column, a vector of size $m$	C^m
$\vec{u}$	Input vector	Vector of size $n$ supplying the coefficients of the linear combination	C^n
$A \vec{u}$	Product	Resulting vector of size $m$	C^m
$\vec{b}$	Vector of constants	Right-hand side of the system $A\vec{x} = \vec{b}$	C^m
${\vec{e}}_{i}$	Standard unit vector	Vector with $1$ in position $i$ and zeros elsewhere	C^n
$C (A)$	Column space	Span of the columns; the set of all reachable products	subspace of C^m
$N (A)$	Null space	Set of vectors annihilated by $A$	subspace of C^n
$S$	Solution set	All $\vec{x}$ satisfying $A\vec{x} = \vec{b}$	subset of C^n

Worked Numerical Example

Problem statement

A three-member truss is analysed by unit-load influence coefficients: entry $[A]_{ij}$ is the force induced in member $i$ by a unit load applied at load point $j$ . Four load points are active with a given load vector $\vec{u}$ in kilonewtons. Compute the member forces two ways and interpret the result.

State the influence matrix and load vector
There are three members and four load points, so $A$ is $3 \times 4$ and $\vec{u}$ has four entries. Conformability holds: one coefficient per column, one output per row.
$A = [\begin{matrix} 2 & - 1 & 0 & 3 \\ 1 & 4 & - 2 & 0 \\ 0 & 3 & 5 & - 1 \end{matrix}], \vec{u} = [\begin{matrix} 3 \\ - 2 \\ 1 \\ 2 \end{matrix}]$
Scale each column by its load
By the definition, the product is $3A_1 + (-2)A_2 + 1A_3 + 2A_4$ . Each term is the member-force pattern produced by one load point acting alone, scaled to its actual magnitude.
$3 [\begin{matrix} 2 \\ 1 \\ 0 \end{matrix}] + (- 2) [\begin{matrix} - 1 \\ 4 \\ 3 \end{matrix}] + 1 [\begin{matrix} 0 \\ - 2 \\ 5 \end{matrix}] + 2 [\begin{matrix} 3 \\ 0 \\ - 1 \end{matrix}]$
Evaluate the scaled columns
Multiply each column entrywise by its coefficient. Note the sign reversal on the second column: a load of $-2$ reverses the entire pattern that load point would otherwise produce.
$[\begin{matrix} 6 \\ 3 \\ 0 \end{matrix}] + [\begin{matrix} 2 \\ - 8 \\ - 6 \end{matrix}] + [\begin{matrix} 0 \\ - 2 \\ 5 \end{matrix}] + [\begin{matrix} 6 \\ 0 \\ - 2 \end{matrix}]$
Add the four vectors
Sum entry by entry. Row 1: $6 + 2 + 0 + 6 = 14$ . Row 2: $3 - 8 - 2 + 0 = -7$ . Row 3: $0 - 6 + 5 - 2 = -3$ .
$A \vec{u} = [\begin{matrix} 14 \\ - 7 \\ - 3 \end{matrix}]$
Check by the row formula
Recompute row 2 independently as a weighted sum along that row: $[A\vec{u}]_2 = (1)(3) + (4)(-2) + (-2)(1) + (0)(2) = 3 - 8 - 2 + 0 = -7$ . This agrees, and because it uses a different traversal it is a genuine check rather than a repetition.
$[A \vec{u}]_{2} = \sum_{k}^{=} [A]_{2 k} [\vec{u}]_{k} = - 7$
Verify superposition on a sub-case
Split the load as $\vec{u} = \vec{v} + \vec{w}$ with $\vec{v} = (3, -2, 0, 0)$ and $\vec{w} = (0, 0, 1, 2)$ . Then $A\vec{v} = (8, -5, -6)$ and $A\vec{w} = (6, -2, 3)$ , and their sum is $(14, -7, -3)$ . Superposition of load groups is exactly linearity in the vector argument.
Recover a column with a unit vector
Applying $A$ to $\vec{e}_3 = (0, 0, 1, 0)$ returns $(0, -2, 5)$ , the third column. In the physical reading, this is a unit test load at point 3 with all other loads removed, and the response is the influence column — which is how such a matrix is measured experimentally.

Result

The three members carry $14$ , $-7$ and $-3$ kilonewtons, with the negative values indicating compression under the stated sign convention. The column reading gives the design insight: member 2 is driven almost entirely by load point 2, so relieving that load point is the effective intervention if member 2 governs. Reading only the final vector would obscure which load point is responsible.

Applications & Industry Use

Structural engineering

Influence coefficients and load superposition

A matrix of unit-load influence coefficients times a load vector gives member forces or displacements directly. The column reading identifies which applied load dominates a given member, which is the information a designer needs to reduce a governing action.

Electrical engineering

Nodal analysis in one equation

Kirchhoff's current law assembled over a network yields $Y\vec{v} = \vec{i}$ , with $Y$ the bus admittance matrix. The matrix-vector product is the entire set of nodal equations, and consistency corresponds to the injected current vector lying in the range of the admittance operator.

Decision analysis and operations

Weighted multi-criteria scoring

A table of alternatives against criteria, multiplied by a vector of criterion weights, produces one score per alternative in a single operation. Negative weights encode undesirable criteria. Changing the weighting is a new product against the same matrix, which makes sensitivity analysis cheap.

Computer graphics

Transforming a point

Applying a rotation, scale or projection to a vertex is a matrix-vector product, and the column reading says the transformed point is a combination of the images of the basis directions. Graphics pipelines exploit this by transforming the three basis vectors once and reusing the columns.

Machine learning

A dense layer forward pass

The affine part of a fully connected neural network layer is $W\vec{x} + \vec{b}$ , a matrix-vector product plus a bias. At inference on a single sample this is the dominant cost and is memory-bound, which is why batching to a matrix-matrix product is the standard optimisation.

Process and chemical engineering

Species balances across a unit

A stoichiometric matrix times a vector of reaction extents gives the net production of every species. The null space of the matrix identifies extent combinations with no net species change, which is the algebraic statement of a closed reaction cycle.

Design Considerations

Keep both readings available

The row formula is the computational recipe; the column formula is the structural explanation. Use the row form to evaluate and the column form to interpret. Practitioners who know only the row form can compute a product but cannot say why a system is inconsistent or which input dominates the output.

Recognise the operation as memory-bound

A dense matrix-vector product performs $2mn$ floating-point operations while reading $m n$ matrix entries, an arithmetic intensity of about two operations per element. That is far below the balance point of modern hardware, so the operation runs at memory bandwidth and no amount of arithmetic tuning helps. Batching many vectors into a matrix product is the standard remedy.

Never form a matrix you only apply

Iterative solvers such as conjugate gradients and GMRES need only the ability to compute $A\vec{u}$ , not the entries of $A$ . Where $A$ arises from a discretised operator, a convolution or a graph, applying it directly can cost $O(n)$ or $O(n \log n)$ against $O(n^2)$ for a dense product, and storage drops from $n^2$ to nothing. This matrix-free approach is standard in large-scale simulation.

Order the loops for the storage layout

The same product can be evaluated by accumulating along rows or by accumulating scaled columns. On row-major storage the row traversal has unit stride and is fast; on column-major storage the column traversal is. Choosing the loop order to match the layout can change throughput by an order of magnitude and is exactly what a tuned BLAS xGEMV does internally.

Exploit structure before optimising the general case

A sparse, banded, triangular, symmetric, circulant or low-rank matrix admits a product far cheaper than the dense $2mn$ operations. A circulant matrix applies in $O(n \log n)$ by FFT; a rank- $k$ factorisation $A = UV^{t}$ applies in $O(k(m+n))$ . Identifying structure is worth more than any micro-optimisation of the dense kernel.

Watch for cancellation in the accumulation

Each output entry is a sum of $n$ terms that may vary widely in magnitude and sign. When the true result is small relative to the largest term, catastrophic cancellation destroys accuracy. Where this matters, use a wider accumulator, order the summation from smallest magnitude, or apply compensated summation; the error bound depends on $\sum |[A]_{ik}[\vec{u}]_k|$ , not on the size of the answer.

Standards & Codes

Notation, interchange and numerical standards that govern how this material is written down, stored and computed in production systems.

Applicable standards, conventions and reference implementations
Reference	Title	Relevance to this topic
`BLAS Level 2`	Basic Linear Algebra Subprograms, matrix-vector operations	The `xGEMV` routine is the reference interface for this operation, with specialised variants `xSYMV`, `xTRMV` and `xGBMV` exploiting symmetric, triangular and banded structure respectively.
`ISO 80000-2`	Quantities and units — Part 2: Mathematics	Fixes the convention that the matrix precedes the vector in the product, that vectors are columns by default, and the bracket notation for extracting entries.
`IEEE 754-2019`	IEEE Standard for Floating-Point Arithmetic	Governs the accumulation of each output entry; the standard's fused multiply-add operation halves the rounding error per term and is used by every tuned implementation.
`ISO/IEC 14882`	Programming languages — C++	Specifies `std::inner_product` and the execution policies used to parallelise the row-wise accumulation, and constrains the reassociation a conforming compiler may perform on the sum.
`MPI Standard`	Message Passing Interface	Distributed matrix-vector products are the core kernel of parallel iterative solvers; the standard's collective reduction and halo-exchange patterns define how the partial sums are combined across processes.

Material Selection

For a mathematical topic, "material" is the numeric representation: the scalar field, storage format and precision the computation is built from.

Representation and precision selection
Representation	Select when	Trade-off
Dense binary64 storage	Moderate matrices with no exploitable structure, where accuracy and simplicity dominate.	$8mn$ bytes and $2mn$ operations per product; simple and predictable but the storage becomes prohibitive beyond a few tens of thousands of rows.
Dense binary32 storage	Inference workloads and graphics where the inputs have limited precision.	Halves memory traffic and so roughly doubles throughput on a bandwidth-bound kernel; accumulation should still be performed in binary64 to limit error growth.
Sparse compressed row storage (CSR)	Finite element, network and graph matrices where most entries are zero.	Storage and work proportional to the non-zero count; indirect indexing prevents full vectorisation and performance depends heavily on the sparsity pattern.
Matrix-free operator	The action of $A$ is defined by a stencil, a convolution or a physical simulation rather than by stored entries.	Storage drops to nothing and the cost can fall to $O(n)$ or $O(n \log n)$ , but no direct factorisation is available and preconditioning becomes the central difficulty.
Low-rank factored form $A = UV^{t}$	The matrix is known or approximated to have rank $k \ll \min(m,n)$ .	Applies in $O(k(m+n))$ instead of $O(mn)$ and stores $k(m+n)$ values; introduces an approximation error that must be bounded against the application's tolerance.
Fixed-point integers	Embedded inference and DSP targets without floating-point hardware.	Deterministic and fast with wide accumulators available in hardware; requires careful scale management per row and risks silent overflow in the accumulation.

Manufacturing Notes

Implementation notes — how the result is actually produced by hand, by algorithm and by library, including cost and numerical behaviour.

Operation count and arithmetic intensity

A dense product costs $m n$ multiplications and $m(n-1)$ additions, about $2mn$ floating-point operations, while touching $mn + n + m$ data elements. The ratio of roughly two operations per element read makes the kernel memory-bound on all current hardware. This is the fundamental reason batched matrix-matrix products, with intensity proportional to the block size, dominate high-performance workloads.

Hand procedure

For hand computation, the column method is more reliable: write the four scaled columns side by side and add across, because the coefficients are used one at a time and stay visible. The row method requires holding a running sum while stepping through both a row and the vector, which is where sign errors enter. Compute by columns and verify one entry by rows.

Loop order and vectorisation

Accumulating by rows gives each output entry a private accumulator and a reduction over $n$ terms; accumulating by columns gives $m$ independent updates per column and no reduction, but writes to the whole output repeatedly. The row form suits row-major storage and vectorises with a horizontal reduction; the column form suits column-major storage and vectorises without one. Tuned xGEMV implementations select between them based on layout and dimensions.

Measuring an unknown linear operator

Because $A\vec{e}_i = A_i$ , a linear black-box system can be identified by exciting each input channel in turn with a unit input and recording the response as a column. This is the mathematical basis for measuring influence coefficients, frequency response matrices and scattering parameters. In practice the excitation is scaled up to overcome noise and the response divided by the amplitude, which linearity permits exactly.

Error behaviour

The computed product satisfies a backward error bound: it is the exact product of a slightly perturbed matrix with the exact vector, with entrywise perturbation of order $n$ times the unit round-off. Forward accuracy is therefore good unless cancellation occurs within an output entry. Using fused multiply-add halves the constant, and using a binary64 accumulator for binary32 data effectively removes the problem.

Failure Modes & Common Mistakes

Failure modes, root causes and prevention
Failure mode / mistake	Impact	Root cause	Prevention & detection
Non-conformable product	high	The vector length does not match the column count of the matrix, often after a transpose or a reshape elsewhere.	Assert that $\vec{u}$ has exactly $n$ entries before the product; remember the result has $m$ entries, so the size generally changes.
Reversing the order to $\vec{u}A$	high	Treating the product as commutative by analogy with scalars.	Keep the matrix on the left and the column vector on the right. If a row-vector-times-matrix form is genuinely wanted, write it as $(A^{t}\vec{u})^{t}$ and be explicit.
Confusing entrywise scaling with the matrix-vector product	high	Using an elementwise multiply, or relying on broadcasting, where the linear combination of columns is intended; in NumPy this is `A * u` versus `A @ u`.	Check the result shape: the product of an $m \times n$ matrix with a size- $n$ vector is a size- $m$ vector, whereas broadcasting returns an $m \times n$ array.
Treating the product as linear in the matrix argument only	medium	Applying superposition to inputs while forgetting the matrix itself must be fixed for linearity to hold.	Superposition applies across load vectors for a single operator. If the matrix changes between cases — a different geometry or a different frequency — the results cannot simply be added.
Forming a dense matrix that is only ever applied	medium	Materialising an operator with $O(n^2)$ storage when an iterative solver requires only its action.	Use a matrix-free operator interface. Reserve explicit assembly for cases needing a direct factorisation, a determinant or an explicit inverse.
Cancellation in an output entry	medium	Summing $n$ terms of similar magnitude and opposite sign, so the result loses most of its significant digits.	Accumulate in higher precision, use fused multiply-add, or reorder the summation. Judge accuracy against the sum of absolute terms, not against the size of the answer.
Mismatched ordering of rows or columns	high	The vector's entries are ordered by a different convention from the matrix's columns, so loads are applied at the wrong points.	Carry explicit labels with both matrix and vector and validate the ordering at assembly time; a permuted input produces a plausible result that no numerical check will catch.
Loop order mismatched to storage layout	low	Traversing columns of row-major storage or rows of column-major storage, so almost every access misses cache.	Match the accumulation order to the layout, or call a tuned `xGEMV` that makes the choice internally.

FAQs

Why define the product as a combination of columns rather than as row dot products?

Because the column form connects the operation to everything already established about linear systems: consistency becomes membership in the span of the columns, and the null space becomes the set of dependencies among them. The row form is an equivalent theorem, better suited to computation, but it hides that structure. Defining by columns and deriving the row formula gives both.

What sizes must the matrix and vector have?

If $A$ is $m \times n$ then $\vec{u}$ must have exactly $n$ entries, one coefficient for each column, and the result has $m$ entries, one for each row. A rectangular matrix therefore changes the size of the vector, which is the algebraic form of a map between spaces of different dimension.

How does

A\vec{x} = \vec{b}

relate to the original system of equations?

The two have exactly the same solution set. A vector $\vec{x}$ satisfies the system precisely when the columns of $A$ , combined with the entries of $\vec{x}$ as coefficients, produce $\vec{b}$ — and that statement is the definition of $A\vec{x} = \vec{b}$ . The vector equation is a re-expression of the system, not an approximation or a reformulation.

Can I recover a matrix from its products?

Yes. Applying $A$ to the standard unit vector $\vec{e}_i$ returns column $i$ , so $n$ products with the unit vectors reconstruct the whole matrix. More generally, agreement of $A\vec{x}$ and $B\vec{x}$ on any basis of $\mathbb{C}^n$ forces $A = B$ , which is the formal statement that a matrix is completely determined by the linear map it induces.

Why is a matrix-vector product slow relative to its operation count?

Because it reads $m n$ matrix entries to perform only $2mn$ operations, and each entry is used exactly once. The kernel therefore runs at memory bandwidth rather than at arithmetic peak. Batching several vectors into a matrix-matrix product reuses each matrix entry many times and moves the workload into a compute-bound regime, which is why batching is the standard optimisation.

Do I always need the matrix entries to compute the product?

No, and often you should not have them. Iterative solvers require only a routine computing $A\vec{u}$ . When $A$ represents a stencil, convolution, graph traversal or physical simulation, applying it directly costs far less than a dense product and needs no storage at all. The trade-off is that direct factorisations and determinants become unavailable, so preconditioning must carry the burden.

Is the matrix-vector product commutative?

No, and the reversed expression is usually not even defined. For $A$ of size $m \times n$ and $\vec{u}$ of size $n$ , the product $A\vec{u}$ exists but $\vec{u}A$ requires $\vec{u}$ to be a $1 \times m$ row. Order is part of the operation, and this asymmetry persists into matrix multiplication.

References

Beezer, R. A. A First Course in Linear Algebra, Version 0.70. University of Puget Sound, 2006. Section MM, Subsection MVP, Definition MVP and Theorems SLEMM and EMMVP. Licensed under the GNU Free Documentation License v1.2.
ISO 80000-2:2019, Quantities and units — Part 2: Mathematics. International Organization for Standardization.
Dongarra, J. J. et al. An Extended Set of FORTRAN Basic Linear Algebra Subprograms. ACM Transactions on Mathematical Software, 14(1), 1988.
Saad, Y. Iterative Methods for Sparse Linear Systems, 2nd edition. Society for Industrial and Applied Mathematics, 2003.
Higham, N. J. Accuracy and Stability of Numerical Algorithms, 2nd edition. Society for Industrial and Applied Mathematics, 2002.

AI Suggested Questions

Given a 3x5 matrix, show me a vector b that is not in the column space and explain what makes the corresponding system inconsistent.
How much faster is a batched matrix-matrix product than 64 separate matrix-vector products on the same matrix, and why?
Write a matrix-free operator for a 2D five-point Laplacian stencil and compare its cost against assembling the sparse matrix.
Construct a matrix-vector product where catastrophic cancellation loses ten significant digits, and show how compensated summation recovers them.
Explain how measuring a frequency response matrix by exciting one channel at a time relies on the identity A e_i = A_i.
Compare the row picture and column picture interpretations of an inconsistent 3x2 system with a sketch of what each says geometrically.

Related Calculators

Matrix-Vector Product Calculator

Evaluate $A\vec{u}$ showing both the column-combination and the row-accumulation routes side by side.

Linear System Solver

Solve $A\vec{x} = \vec{b}$ , report consistency, rank and the vector form of the solution set.

Column Space Membership Tester

Decide whether a given $\vec{b}$ lies in $\mathcal{C}(A)$ and, if so, return the coefficients that produce it.

Overview

Definition

Matrix-Vector Product

Systems of Linear Equations as Matrix Multiplication

Equal Matrices and Matrix-Vector Products

Concepts

The column picture and what it reveals

The row picture and the connection to inner products

Linearity of the product in the vector

Standard unit vectors extract columns

The null space in product form

A matrix is determined by its action

Decision path: evaluating and interpreting Au→

Equations

Variable Definitions

Worked Numerical Example

Problem statement

State the influence matrix and load vector

Scale each column by its load

Evaluate the scaled columns

Add the four vectors

Check by the row formula

Verify superposition on a sub-case

Recover a column with a unit vector

Applications &amp; Industry Use

Influence coefficients and load superposition

Nodal analysis in one equation

Weighted multi-criteria scoring

Transforming a point

A dense layer forward pass

Species balances across a unit

Design Considerations

Keep both readings available

Recognise the operation as memory-bound

Never form a matrix you only apply

Order the loops for the storage layout

Exploit structure before optimising the general case

Watch for cancellation in the accumulation

Standards &amp; Codes

Material Selection

Manufacturing Notes

Operation count and arithmetic intensity

Hand procedure

Loop order and vectorisation

Measuring an unknown linear operator

Error behaviour

Failure Modes &amp; Common Mistakes

FAQs

Related Engineering Topics

References

AI Suggested Questions

Related Calculators

Continue learning

Decision path: evaluating and interpreting $A\vec{u}$

Applications & Industry Use

Standards & Codes

Failure Modes & Common Mistakes