KEVOS® MAKEOS · Quality
Process Change Trial (Sample Size, Welch t-Test, Improvement Decision)
How many parts should I run before and after a process change to detect the improvement I care about, and do my trial results prove the change made a real difference?
Planning and evaluation of a before-and-after trial of a process change: the sample size per group to detect your smallest worthwhile difference at the chosen significance and power (equivalent to the sample size for a mean tool with the margin set for a difference), and the Welch two-sample t-test on the results with p-value and confidence interval (as the two-sample t-test tool), then the decision: improvement confirmed, no proven difference, or made worse, with checks of sample size and practical size of the change.
Open the calculatorRuns in your browser. No account needed.
How this tool works
Inputs, units and where to find them
| Input | Unit | Typical source |
|---|---|---|
| Plan | ||
| Expected standard deviation | Standard deviation you expect, from process history. | |
| Smallest difference worth detecting | Smallest change in the mean that would matter to you. | |
| Significance level (two-sided) | choice | Chance you accept of claiming a difference that is not there. |
| Power | choice | Chance of detecting a real difference of the size you set. |
| Better is | choice | Whether a lower or a higher result is an improvement. |
| Before the change | ||
| Mean | Mean of the parts made before the change. | |
| Standard deviation | Standard deviation before the change. | |
| Sample size | items | Parts measured before the change. |
| After the change | ||
| Mean | Mean of the parts made after the change. | |
| Standard deviation | Standard deviation after the change. | |
| Sample size | items | Parts measured after the change. |
Method and formulas
- n per group = 2 [(z_α/2 + z_β) σ ÷ δ]², rounded up — the same as the sample size for a mean with margin δ ÷ [√2 (1 + z_β ÷ z_α/2)].
- Welch t = (x̄₁ − x̄₂) ÷ √(s₁²/n₁ + s₂²/n₂) with Welch–Satterthwaite degrees of freedom; two-sided p-value and confidence interval (as the two-sample t-test tool).
- Improvement confirmed if p < α and the difference is in the better direction.
- Example: plan 19 parts per group; the trial shows a difference of 2.2 with p = 0.020303.
Limitations
- Independent, roughly normal measurements; trends over time within a group invalidate the test.
- A significant result shows a difference in means only, not a change in spread.
- Example values are illustrations. Replace them with values for your actual task.
Typical uses
- Plan a trial of a new tool, material or setting
- Decide whether a change really improved the process
- Avoid acting on chance differences
Who it is for
- Quality engineers
- Process engineers
- Continuous improvement leads
What you get
- Sample size per group for the trial
- t statistic, p-value and confidence interval
- Decision and checks
Interpreting the result
- A difference can be statistically significant yet too small to matter: compare it with the difference you set as worthwhile.
References
- Machinery’s Handbook, 29th edition (Industrial Press, 2012), printed pp. 130–133. Topic reference; no source tables or recommended settings are reproduced.
WORKOS tools are screening and decision-support utilities. Verify important financial, engineering, safety, legal or operational decisions using appropriate professional standards and your organisation’s approved data.
Open the Process Change Trial (Sample Size, Welch t-Test, Improvement Decision) calculator
