§1Accuracy, precision, resolution
Three different virtues, routinely confused: accuracy is closeness to truth, precision is repeatability, resolution is merely the fineness of the readout.
A digital calliper resolving 0.01 mm is not accurate to 0.01 mm; a worn micrometer can repeat beautifully around the wrong answer. The working discipline is the 10 : 1 rule: the instrument’s uncertainty should be about a tenth of the tolerance it polices — a 0.02 mm tolerance wants a micrometer, not a calliper; a 21 µm IT7 zone (the fits page’s hero) wants better than a 2 µm instrument. Where 10 : 1 is unaffordable, 5 : 1 with statistical backup (the Manufacturing Data Analysis page) is the honest fallback.
Contents§2Rules and callipers
The rule is for ±0.5 mm work; the calliper — vernier, dial or digital — earns its keep in the ±0.05 mm band across outside, inside, depth and step in one tool.
The vernier’s trick: a sliding scale of 50 divisions spanning 49 main-scale half-millimetres, so each division differs by 0.01 mm — the reading is the main scale at the zero plus the vernier line that aligns. Its lies are mechanical: jaw wear at the tips (measure deep in the jaws when possible), Abbe offset (the scale is not in line with the jaws, so a cocked slide reads wrong) and feel — callipers have no ratchet, so pressure is the operator’s calibration. Digital removes the reading error and none of the mechanical ones.
Contents§3Micrometers
A precision screw of 0.5 mm pitch turns rotation into measurement: fifty thimble divisions per turn make each division 0.01 mm, honestly.
Sleeve shows 12.5 mm exposed; thimble line 28 against the datum: reading = 12.5 + 0.28 = 12.78 mm. A vernier sleeve adds a micron digit for those who trust their temperature control.
Habits that keep the micron honest: use the ratchet or friction thimble for constant force; check zero (or a setting standard for larger frames) before a batch; clean the faces on paper; and never leave the faces clamped shut — steel needs its expansion room. Frame ranges step by 25 mm because the screw stays short and stiff; the anvil grows instead.
Contents§4Dial and test indicators
Indicators measure change, not size — runout, flatness, squareness, machine setup — and the lever-type carries a geometric lie worth pricing.
A test indicator run with its stylus 20° off the surface over-reads by 1/cos 20° − 1 = 6.4 % — a 0.10 mm runout displays as 0.106. At 10° the lie shrinks to 1.5 %. Keep the stylus within ~10–15° of parallel, or multiply by cos θ when geometry forbids it.
Plunger-type dial indicators avoid cosine error along their axis but obey their own rules: mount rigidly (a wobbling magnetic stand measures the stand), approach readings in one direction to sink backlash, and remember the reading is only as square as the plunger is to the motion measured.
Contents§5Gauge blocks
The workshop’s embodiment of the millimetre: hardened, lapped blocks that wring together into any length, micron-true.
Eliminate the finest digit first: take 1.005 (leaves 26.48), then 1.48 (leaves 25), then 5 + 20. Stack = 1.005 + 1.48 + 5 + 20 = 27.485 mm in four blocks — fewest blocks, fewest wringing films. Wrung joints are so intimate their film is nanometres; blocks are graded (calibration, inspection, workshop) by permitted deviation, and a workshop set checks the instruments that check the parts.
§6The sine bar
Angles are manufactured from lengths: a bar of exact roller spacing L, propped on gauge blocks of height H, tilts at a computable angle.
H = 200 × sin 5° = 17.431 mm — a stack the §5 method builds in moments. The sine bar’s honesty degrades as θ grows (the sine flattens, so a block error buys more angle error); keep it below about 45° and prefer the complement setup beyond. The Solution of Triangles page carries the trigonometry both ways.
§7Transfer and bore measurement
Some features can’t meet a micrometer directly; the measurement is captured, then carried to one.
Telescoping gauges spring to a bore’s diameter, lock, and are measured outside — cheap and honest with practised feel, two readings recommended. Three-point bore micrometers centre themselves and read directly, and being three-point they also expose lobing that a two-point measurement across an odd-lobed bore cannot see — a part can be “round” to a two-point check and still not fit its shaft. Small-hole gauges cover the sizes below telescoping range. For form and position beyond size — concentricity, flatness, true position — the work moves to a surface plate with indicators, or to a coordinate measuring machine, where the inspection plan and datum scheme come straight off the drawing’s datums (Drafting Practices, §5).
Contents§8Temperature and the 20 °C convention
Every dimension in this Library is a 20 °C dimension by international convention — because steel is a different size at every other temperature.
A 300 mm steel part measured at 26 °C is longer by 300 × 11.7 × 10⁻⁶ × 6 = 21 µm — coincidentally the entire IT7 tolerance at Ø25 from the fits page. Handling heat does it too: minutes in a warm hand moves microns. The defences: let part and instrument soak to the same temperature (like materials then largely cancel), handle precision work with gloves or tongs, and measure to microns only in controlled rooms.
§9Inspection strategy
What to check, how often, and against what authority — the plan matters as much as the instrument.
First-article inspection proves the setup before the batch runs; in-process checks at a planned frequency catch drift while it is still adjustment rather than scrap — the control-chart logic of the Manufacturing Data Analysis page; final inspection samples or screens per the part’s consequence of failure. Underneath all of it runs traceability: workshop instruments checked against reference standards, reference standards calibrated up an unbroken chain to national standards, each link with known uncertainty and a due date. An uncalibrated instrument doesn’t measure — it opines.
Contents§10Quick reference
The working core of the page on one card rack.
Rule of thumb
instrument ≈ tolerance / 10
Micrometer
0.5 mm pitch · 50 div = 0.01
ratchet for constant force
Cosine error
true = reading × cos θ
20° ⇒ +6.4 %
Sine bar
H = L sin θ
200 mm · 5° → 17.431
Temperature
steel 11.7 µm/m/°C
all sizes are 20 °C sizes
Strategy
first article → in-process → final
no calibration, no measurement