The notation, cost model and complexity conventions used throughout this collection, including the sub-exponential L-function.
Engineering / MathematicsOrientation9 min readKV-MATH-0502
Cost statements in computational number theory are only meaningful relative to a stated model. This page fixes the conventions used everywhere else in the collection.
The cost model
Costs are counted in bit operations unless stated otherwise. The alternative — counting arithmetic operations on machine words — hides exactly the growth that dominates in this subject, because the numbers involved routinely exceed a machine word by orders of magnitude.
Conventions
Input sizeThe number of bits, written n, not the value
MultiplicationM(n) denotes the cost of multiplying two n-bit integers
DivisionAssumed O(M(n)) up to constants
MemoryCounted when it is the binding constraint, which for sieving methods it usually is
Asymptotic notation
O(f(n))
Bounded above by a constant multiple of f(n) for large n. The default in this collection.
Omega(f(n))
Bounded below similarly.
Theta(f(n))
Bounded both above and below.
o(f(n))
Grows strictly slower than f(n).
Soft-O
Written with a tilde; suppresses logarithmic factors. Useful when those factors are not the point.
The sub-exponential L-function
Integer factorisation and class group computation both run in time that is neither polynomial nor exponential in the input size. The standard notation for this middle range is:
L_N(a, c) = exp( (c + o(1)) (log N)^a (log log N)^(1-a) )N is the number being processed; a lies in [0,1]; c is positive.
The parameter a is what matters. It interpolates between the two familiar regimes and gives a single scale on which to compare methods.
Reading the parameter a in L_N(a, c)
Value of a
Regime
Example
a = 0
Polynomial in log N
Modular exponentiation
a = 1/2
Classic sub-exponential
Quadratic sieve, ECM, class group methods
a = 1/3
Improved sub-exponential
Number field sieve
a = 1
Exponential in log N
Trial division
Probabilistic and conditional results
Three qualifiers appear repeatedly and are not interchangeable.
Las Vegas
Always correct when it terminates; running time is random. Cantor-Zassenhaus splitting is of this type.
Monte Carlo
Fixed running time; may return a wrong answer with bounded probability. A single Fermat test is of this type.
Conditional on GRH
Correctness or complexity depends on an unproven hypothesis. The result is real but its status differs from an unconditional one.
Heuristic complexity
Several headline complexities in this subject are heuristic: they rest on plausible but unproven assumptions about how often values produced by an algorithm are smooth. The number field sieve's L(1/3) bound is of this kind. These estimates match observed behaviour closely, which is why they are quoted, but they are not theorems.
Frequently Asked Questions
Why count bit operations rather than machine operations?
Because the integers involved are much larger than a machine word, so the cost of a single 'operation' varies with the size of its operands. Counting word operations would make a 10-digit and a 10,000-digit multiplication look equally cheap.
What does o(1) mean inside the L-function?
It absorbs terms that vanish as N grows. It is the reason L-function complexities should be read as asymptotic shapes rather than as formulas you can substitute into for a specific N.
Source. Henri Cohen, A Course in Computational Algebraic Number Theory, Springer GTM 138 — 1.1. Structural reference unverified: the source file was not available during authoring; chapter and section numbers are taken from the published edition and have not been checked against a physical copy.
Handbook application: from concept to controlled practice
Purpose. This expanded section turns the original page into a practical handbook. It preserves the supplied material and adds a repeatable way to apply, check and review Algorithm Notation and Complexity Conventions. It does not replace a contract, legislation, a controlled standard, competent engineering judgement or specialist advice.
The operating aim is to turn a compact mathematical statement into a usable chain of definitions, claims, examples and checks. Read the original explanation first, then use the workflow and checks below to convert knowledge into evidence.
Treat Algorithm Notation and Complexity Conventions as a network of definitions and implications, not as a list of formulas. The working vocabulary on this page—notation, complexity, cost, model, sub-exponential—should be made explicit before any proof or computation begins. Record the ambient set or structure, the permitted operations and the equality or equivalence relation in use. A compact theorem often changes meaning when the base field, finiteness condition, commutativity assumption or direction of an action changes.
For a proof, write the hypotheses as a checklist and mark the line at which each one is used. For a computation, state the representation of the input, the arithmetic model, the termination condition and the output invariant. For a classification problem, distinguish existence from uniqueness and distinguish an object from its representation. These separations prevent a correct local calculation from being mistaken for the general result.
A useful worked example should be small enough to inspect completely but rich enough to exercise the main mechanism. Compute the result in two ways where practical: symbolically and by substitution, structurally and numerically, or directly and through a normal form. Then include one near-miss example in which a hypothesis fails. The contrast explains why the theorem is shaped as it is and gives the reader a diagnostic pattern for later problems.
Verification is part of the mathematics. Check domains and codomains, substitute proposed solutions, test identity and zero cases, compare dimensions or cardinalities, and confirm that maps respect the required operations. In numerical work, report precision, conditioning and a residual rather than digits alone. In algorithmic work, separate mathematical correctness from implementation complexity and resource limits.
Step-by-step operating method
Fix the setting. State the objects, ambient structure, notation and assumptions before manipulating symbols.
Separate claims. Distinguish definitions, hypotheses, conclusions, equivalent conditions and consequences.
Choose a method. Select proof, construction, calculation or algorithm according to the question actually asked.
Work a small case. Use the smallest non-trivial example to expose the mechanism and test edge behaviour.
Verify independently. Substitute back, check invariants, test boundary cases or use an alternative derivation.
Worked-example protocol
Illustrative method—not a source theorem. Start with a small admissible input and list the definitions it must satisfy. Carry out each transformation on a separate line, citing the property that permits it. Preserve exact values until approximation is necessary. At the end, verify the output against the original definition and one invariant such as dimension, degree, determinant, order, norm or residual. Then alter one hypothesis and observe which step ceases to be valid. This protocol creates a reusable example without inventing a theorem-specific numerical answer.
Stage
Record
Quality check
Input
Objects, domain, notation, assumptions
Every symbol is defined
Method
Permitted operation or cited result at each step
All hypotheses hold
Output
Exact result and representation
Correct type, domain and form
Verification
Substitution, invariant or alternative derivation
Independent agreement
Boundary test
Zero, identity, degenerate or failed hypothesis
Scope is understood
Common failure modes and recovery actions
1. Watch for
Using a theorem without checking every hypothesis.
Recovery: Return to the governing definition or requirement and restate the decision in one sentence.
2. Watch for
Treating a suggestive example as a proof of the general case.
Recovery: Separate evidence from assumption, assign an owner and set a date for validation.
3. Watch for
Changing notation or conventions part-way through an argument.
Recovery: Run a small counterexample, boundary test, pilot or independent check before proceeding.
4. Watch for
Hiding a division-by-zero, convergence, finiteness or commutativity assumption.
Recovery: Record the consequence, decision and rationale, then update the controlled baseline.
5. Watch for
Reporting a computed result without a residual, substitution or structural check.
Recovery: Escalate when the issue affects safety, compliance, acceptance, material value or an agreed tolerance.
Review checklist
Can every symbol be traced to a definition or prior result?
Which hypothesis does each major step use?
Does the method cover zero, identity, degenerate and boundary cases?
Can the conclusion be checked by a second representation or calculation?
Are mandatory requirements distinguished from recommendations and illustrative values?
Are sources, assumptions, units, dates and versions recorded closely enough to reproduce the decision?
Have safety, legal, ethical, stakeholder and operational consequences been considered at the appropriate level?
Is there a named owner and a trigger for review, escalation, change or retirement?
Questions for deeper application
What is the most important distinction a practitioner must preserve when applying Algorithm Notation and Complexity Conventions?
Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.
Which assumption about notation would change the result most if it proved false?
Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.
What evidence would allow an independent reviewer to reproduce or challenge the conclusion?
Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.
Which boundary, exception or failure case has not yet been tested?
Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.
What must be handed over, monitored or reviewed after the immediate work is complete?
Answer with a fact or cited source where available. Where evidence is incomplete, record the assumption, consequence, responsible owner and next validation action.
Authoritative references and use notes
The sources below were selected as institutional or primary guidance for the broader practice. They support the handbook method; they do not imply that every statement or clause in a source applies to every project. Confirm the current edition, jurisdiction, contract and application before treating any requirement as mandatory.
NIST Digital Library of Mathematical Functions — National Institute of Standards and Technology. Used for mathematical notation, numerical methods, asymptotics and special functions. Accessed 2026-08-13.
MIT OpenCourseWare — Number Theory I — Massachusetts Institute of Technology. Used for algebraic and analytic number theory. Accessed 2026-08-13.
