Choose your decision measures before the design: why criteria belong at the start

Many businesses measure the consequences of a design after the important decisions have been made. A product is designed, a process chosen, equipment sized, suppliers appointed and money committed. Only then does someone calculate the running costs, the waste, the energy use, the maintenance burden or the effect on customers. By that point, measurement can describe the result but can no longer change it.

The alternative is to treat the measures themselves as part of the design. Before options are narrowed, decide which forms of value and risk matter: technical performance, whole-of-life cost, environmental effects, effects on customers, staff and the community, and the risk and reversibility of the choice. Those criteria then shape which options are developed, rather than merely scoring the one that was chosen.

This article explains why the timing of measures matters, how to choose a small set of measures that actually improve decisions, how to combine thresholds, objectives and judgement without hiding trade-offs in a single score, and how a small business can apply the approach to products, processes and investments.

Measures decide what a business can see

What a decision process measures determines what it can see. What it can see influences which options survive. Which options survive determines what is eventually built. If an investment case contains purchase price and output but not maintenance, waste, disposal or effects on customers, those consequences do not disappear. They move outside the decision, and the business, its customers or the community pay for them later.

An effect that falls on someone outside the decision, without being counted in it, is sometimes called an externality. Some externalities are environmental, such as pollution. Many are closer to home: a design that is cheap to make but slow to install shifts cost to installers; a process that saves operating cost but is hard to maintain shifts cost to the maintenance team; a product that is cheap to buy but hard to repair shifts cost to customers.

Why timing matters

Early in a project, at the concept stage, a business can still change the basic architecture: the process route, the materials, the location, the energy source, the product format, the supplier model. Later, the same change may require redesign, contract variations or replacing equipment. Measures applied early can redirect the design. Measures applied late can only optimise around the edges.

That is why early measures are valuable even when the information behind them is rough. Their job at the concept stage is not to produce a precise forecast. It is to show which directions deserve closer investigation before irreversible choices are made. Rejecting an option at concept stage because early estimates are imprecise misunderstands their purpose.

What the research suggests

Published research on resource recovery and engineering design supports this view. A review of measures used to assess recovering resources from waste warned that focusing on one or only a few dimensions can give misleading messages, and argued for considering environmental, economic, social and technical value together, including who gains and who bears the costs, and how that changes over time. It also stressed that measures must remain simple, transparent and specific enough to be useful. Research on recycled concrete aggregate found that technical, financial or environmental assessments on their own could not answer whether the recycled option should be preferred, so the authors combined them. Work in chemical process design proposed bringing energy, material and risk measures into the concept stage, precisely because that is when the design can still change cheaply.

Materials research also shows why technical qualification belongs in the measure set. A study of road fill made from treatment-plant biosolids and fly ash found that material from two different treatment plants behaved differently, and tested chemical leaching alongside load-bearing strength, because strength alone was not enough to show the material was fit for use.

Five domains of value

A practical measure set covers five domains:

Not every domain matters equally for every decision. A packaging decision, a factory layout and a software system need different measures, even if some business-wide criteria are shared. Build a short measure set for each type of decision rather than one universal scorecard.

Thresholds, objectives and judgement

Not every measure should be treated the same way:

• Thresholds are requirements an option must meet to be considered at all: safety, legal compliance, minimum performance. They are not traded against other benefits.
• Objectives are measures where more or less is better, such as lower cost per unit or less waste, and where options can be compared and traded.
• Judgement criteria are considerations that matter but resist measurement, such as fit with brand or effect on a key relationship. They should be stated and discussed, not hidden.

Avoid collapsing everything into one composite score too early. A single number is convenient, but it can hide that one option is technically stronger but riskier, or cheaper but harder on customers. Look at the separate dimensions first. If a combined score is used, make the weighting visible and test whether reasonable changes to it would change the answer.

Five tests for each measure

More measures do not automatically mean better decisions. A dashboard with dozens of indicators can be as misleading as one with a single number, because attention spreads thin and it becomes easy to pick the measure that supports a preferred answer. Test each proposed measure:

1. Material: could it change the decision?
2. Causal: does it describe a real consequence, or a convenient proxy?
3. Transparent: can decision-makers understand how it is calculated?
4. Proportionate: is the effort of measuring it justified by the size and reversibility of the decision?
5. Stakeholder-aware: does it capture where value and burden actually fall?

Measures that fail these tests should be challenged rather than accumulated. Watch especially for measures chosen because data is easy to obtain. What is easy to measure is not always what matters.

Set the boundary deliberately

Every measure depends on a boundary: which costs and effects are included, for whom, and over what period. Boundaries are often set by accident. A project budget covers purchase and installation, so running costs fall outside it. A department measures its own efficiency, so effects on other departments are ignored. A product cost covers manufacture, so freight, installation and returns are left out.

Decide the boundary consciously and write it down. Useful questions:

• Time: over what period are costs and benefits counted? The purchase, the first year, the expected life?
• Stages: are supply, manufacture, delivery, installation, use, maintenance and disposal all included?
• People: whose costs and benefits count? Only the business, or also customers, installers, staff and neighbours?
• Uncertainty: which estimates are firm, and which are rough?

Two options compared within different boundaries cannot be compared fairly. Use the same boundary for every option in a decision.

Revisit measures without moving the goalposts

Measures should improve as a project learns more. A threshold may need refining once a regulation is checked, or an objective may prove impossible to estimate and need replacing. But changing measures midway carries a risk: the measures can drift to favour whichever option someone already prefers. A simple safeguard is to record every change to the measure set, with the reason, and to check whether the change alters the ranking of options. If it does, discuss that openly before proceeding.

A simple decision-measure sheet

For a significant decision, a one-page sheet prepared before options are developed is often enough. It records:

• The decision and its scope.
• Thresholds every option must meet.
• Three to six objectives, with how each will be estimated.
• Judgement criteria to discuss.
• Who is affected, inside and outside the business.
• The boundary: which costs and effects are included, and over what time.
• How reversible the decision is, and therefore how much evidence it needs.

The sheet is then used to develop options, not just to score them at the end.

A worked example

This is an illustration. A small business plans a new range of outdoor storage boxes for homes and trades. In past projects, the team designed a product and then costed it. This time the owner writes a decision-measure sheet first.

Thresholds: weather resistance suitable for outdoor use, a defined lid load, no pinch points and a lockable lid. Objectives: low landed cost per unit including tooling over the first 2,000 units, low freight volume, and assembly time under 20 minutes if customers assemble it. Environmental: material per unit and whether the product can be repaired or recycled. Stakeholders: customer assembly effort and retailer storage space. Risk: tooling commitment and supplier dependence.

The team develops three concepts. All figures are illustrative.

Judged on manufacturing cost alone, which is how the business had decided before, the moulded box wins. Once freight and tooling are counted, the steel flat-pack is cheapest landed by a clear margin, takes much less retailer storage space and carries far less tooling commitment. The timber composite option is held back until its weathering is tested. The assembly time for the steel box is within the 20-minute objective, and the team designs the parts so they can only fit together one way.

Because the measures were set first, the design team spent its effort making the flat-pack design easy to assemble, rather than discovering at the end that freight made its preferred design uncompetitive.

How this applies to a small Australian business

Small businesses rarely have formal design governance, which makes a simple early discipline more valuable:

• Write the decision-measure sheet before developing options, not after.
• Include the full life: purchase, freight, installation, operation, maintenance and disposal.
• Ask who else is affected: customers, installers, retailers, staff and neighbours.
• Keep the measure set short: three to six objectives plus thresholds.
• Make thresholds explicit, particularly safety and legal requirements, and confirm them with the relevant regulator or adviser.
• Revisit the sheet when new evidence arrives, but be wary of changing measures to suit a preferred option.

The articles on avoiding burden shifting in low-carbon decisions and the cost you commit before you spend cover related ideas.

Signals worth watching

• Assessments that begin after the preferred option is chosen.
• Proposals that claim improvement on one measure only.
• Composite scores with weightings nobody can explain.
• Measures that cannot be linked to any decision.
• Costs excluded because they fall outside the project budget.
• The preferred option changing when one omitted measure is added.

The last signal is the most important. If adding a missing measure would change the answer, the measure set, not the engineering, is deciding the outcome.

Common mistakes

• Measuring after deciding.
• Adding too many measures and diluting attention.
• Letting available data define the decision.
• Combining unlike values too early into one score.
• Treating thresholds as trade-offs, especially for safety.
• Ignoring who bears the costs outside the business.
• Rejecting early options because concept-stage estimates are imprecise.

Frequently asked questions

Is this only relevant to sustainability? No. The same approach applies to any significant decision. Environmental measures are one domain among five.

How precise do early estimates need to be? Only precise enough to distinguish options and reveal which directions deserve more work. Precision improves as the design develops.

Who should set the measures? The people accountable for the outcome, with input from those affected: operations, maintenance, sales, customers or installers as relevant.

Questions to ask

• Which forms of value are currently invisible in this decision?
• Are our measures set early enough to change the design?
• Which measures are real outcomes, and which are convenient proxies?
• Which requirements are thresholds rather than trade-offs?
• Who benefits, and who bears costs or risks outside the business?
• Would a simpler set of measures give an equally good decision?
• Would our preferred option change if we added one missing measure?

Bringing it together

The measures a business chooses before design determine which forms of value it can see, protect and create. Set them at the concept stage, when the design can still change. Cover technical, financial, environmental, stakeholder and risk dimensions, but keep the set short and test each measure for materiality, causality, transparency, proportion and stakeholder awareness. Separate thresholds from objectives and judgement, avoid hiding trade-offs in a single score, and use the measures to develop options, not just to grade them. The best time to design the measures is before the design becomes expensive to change.

---

Source: KEVOS notes, drawing on published research on multi-dimensional value in resource recovery, integrated assessment of recycled aggregate concrete and sustainability indices in process design. Figures in this article are illustrations.