01Executive summary
Two recent milestones, included for their engineering content rather than because their historical weight is settled: computing became a utility, and pattern recognition became a matter of scale.
From around 2006, computing capacity became purchasable by the hour without owning hardware. In 2012 a deep convolutional network trained on graphics processors substantially outperformed prior methods on a large image classification benchmark, redirecting a field that had spent decades on hand-designed features. Both are recent enough that this series makes no claim about their eventual standing.
02Utility computing: an old idea that needed conditions
Computing as a metered utility was proposed in the 1960s and time-sharing services existed for decades. What changed in the mid-2000s was not the concept but the conditions: cheap commodity servers, mature hardware virtualisation, abundant network bandwidth, and organisations operating data centres at a scale where the fixed costs of automation paid back.
Owning hardware means sizing for peak, paying upfront, and carrying the risk of getting the forecast wrong in either direction. Renting means paying for what is used and moving the forecasting risk to a provider who aggregates many uncorrelated demand curves. That aggregation is the same argument as pooled electricity reserve in the previous series — diversity across many users smooths total demand, so shared capacity serves everyone with less total plant than each would need alone.
Utility computing is not uniformly cheaper. Steady, predictable, high-utilisation workloads are frequently more expensive rented than owned, and data transfer charges and managed service premiums accumulate quietly. There is also a real dependency: infrastructure defined against one provider’s interfaces is expensive to move. The engineering judgement is about demand variability and the value of speed, not about a general cost advantage — and it should be made on measured utilisation rather than on the assumption that renting is modern and owning is not.
03Learned features: what actually changed in 2012
Recognising objects in images was for decades approached by designing features — mathematical descriptions of edges, corners, gradients and textures that a human researcher believed were informative — and then training a classifier on those. Progress was steady and slow, and the difficult part was the feature design.
The 2012 result was not a new idea. Convolutional networks had existed since the 1980s and backpropagation longer than that. What changed was that three things arrived together: a labelled dataset large enough to train a high-capacity model without it simply memorising, graphics hardware whose dense parallel arithmetic suited the computation, and a set of practical training techniques that made deep networks converge reliably.
Old idea, new conditions
As with utility computing, the concept long predated the milestone. Recognising that an idea has been waiting for enabling conditions — rather than being wrong — is a genuinely valuable engineering judgement.
Hardware chosen for the wrong reason wins
Graphics processors were built to shade pixels. Their suitability for training was incidental, and it reshaped an industry. Adjacent capability frequently matters more than purpose-built capability.
The data became the specification
Behaviour is determined by the training set rather than written down. Dataset composition, labelling quality and coverage become engineering artefacts requiring version control and review.
Verification became genuinely hard
A learned function has no readable specification to check against. Assurance rests on statistical performance over held-out data, which does not bound behaviour on inputs unlike anything in the training set.
The functional safety framework covered later in this series assumes a system has a specification against which correctness can be argued. A learned component does not, in the usual sense. The engineering response has been to constrain what such components are permitted to decide, to bound them with conventional monitors that do have specifications, and to treat the training and validation data as controlled configuration items. This is an active and unsettled area, and any account claiming it is resolved should be treated with suspicion.
04Takeaways for current practice
- Ask whether an old idea is wrong or merely early. Both of this part’s milestones were decades-old concepts that needed enabling conditions.
- Buy elasticity, not cheapness. Rented capacity earns its premium on variable demand and speed of provisioning, not on unit cost.
- At scale, tolerate failure rather than prevent it. Continuous component failure inverts conventional reliability practice.
- Treat training data as a controlled configuration item. Where the data is the specification, it needs version control, review and coverage analysis.
- Bound components you cannot specify. Constrain their authority and monitor them with something that does have a specification.