Non-Response and Sampling Bias
Four sentences on a single slide carry the only complete piece of causal reasoning in the week — and they hold up. The material around them names a second bias in a different document, undercuts its own claim about randomisation, and offers no remedy at all.
The slide, in full
Bias gets one slide in the quantitative research week. It is titled "Sampling bias" and it runs to four sentences, quoted here complete because nothing else in the deck adds to them.
Two incidental things are worth noticing before the argument itself. This is the first point in the week at which the data are said to come from people who answer questions rather than from units that are measured — nothing earlier says so. And "respondents" enters here undefined, as the fifth word the deck has used for the unit of analysis, after subjects, people, items and objects.
The causal chain, and why it is the best thing in the week
Most of this week's content is a list of labels. This slide is not: it states a mechanism, follows it to a change in the composition of the data, and follows that to a directional consequence for the result. The chain is correct as reasoned, and it is the only complete one in the week.
- Respondents choose whether to answer
- Those who decline differ in their views
- The returns over-represent one view
- The estimate moves in a predictable direction
The source's illustration, taken apart
The mechanism
Satisfied respondents do not think the survey is worth the trouble, so they do not complete it. Nothing is done to them; they select themselves out.
The resulting composition
The returns therefore contain proportionally more dissatisfied respondents than the population does. The sample is no longer a small version of the group it came from.
The directional consequence
Estimates of overall satisfaction come out "lower than they should be". The direction is stated and the size of the error is not — no response rate, no sample size, no magnitude appears anywhere on the slide.
The mechanism transfers directly to the work this library's readers do. A post-implementation survey circulated to everyone who touched a delivery will be answered disproportionately by the people with something to say about it. So will a lessons-learned questionnaire sent after a difficult programme. That application is this library's extension of the source's reasoning rather than the source's own example, and it is worth making explicitly because the source stops at the mechanism and never brings it near an organisational setting.
The sentence with no antecedent
The slide opens "The most common is non-response." The most common what? The title supplies "sampling bias" and nothing on the slide completes the phrase.
Two kinds of bias, one per document, never related
The week names two biases, one in each of its two documents, and neither document mentions the other's. The distinction matters because the two sit at different moments in a study: one is about who gets approached, the other about who then answers.
THE WEEK'S TWO NAMED BIASES AND THEIR REGISTERS
| Term | Where it appears | Register | What is given |
|---|---|---|---|
| Non-response bias | The slide quoted above | The subject's own voice — teaching material | A mechanism, a resulting composition and a direction of error. No definition of the term itself, and no remedy |
| Selection bias | The study notes | A published article in sport and exercise science, reproduced word for word | "When the sample is not representative of the population, selection bias is a possibility", plus a formal definition of a biased statistic |
| An unnamed third | The first sampling-types slide | The subject's own voice | Only an allusion: random selection "avoids biases that enter when trying to select a representative sample". Those biases are never named |
Three references to bias, two named kinds, one formal definition, and no point anywhere in the week at which they are related to one another.
The formal definition is worth having, provided you attribute it correctly. In the borrowed text, a statistic is biased "if the value of the statistic tends to be wrong (or more precisely, if the expected value — the average value from many samples drawn using the same sampling method — is not the same as the population value)". That is the only formal definition of bias anywhere in the supplied material, and it is not in the subject's own voice.
The claim about randomisation, and the slide that undercuts it
Three slides before the bias slide, the deck credits random selection with avoiding "biases that enter when trying to select a representative sample". The bias slide then describes something randomisation does not touch, and neither slide qualifies the other.
What the earlier slide claims
- Random selection is "chosen as the result of some random or chance process".
- Generally all objects in the population have an equal chance of being chosen.
- It "avoids biases that enter when trying to select a representative sample".
- The claim is stated without conditions and without naming the biases avoided.
What the bias slide then says
- Respondents choose whether or not to fill in the form or take part in an interview.
- That choice is available to a randomly selected respondent exactly as it is to any other.
- If the decliners differ in their views, the result is biased regardless of how they were selected.
- The slide does not mention randomisation, and the randomisation slide does not mention response.
Reported side by side, the two claims cannot both stand as stated. This library does not adjudicate between them, and the same unqualified claim about randomisation is recorded in the later data collection week, so it is not a slip on one slide. If you rely on random selection in your design, describe what it did for your selection and address response separately — the material will not do that separation for you. The broader treatment of designing bias out is at controlling bias, which draws on the set reading rather than the deck.
There is a second connection the deck leaves unmade. Self-selected sampling is one of the three methods listed in its non-random category, and self-selection into the response is precisely the mechanism the bias slide describes. The two slides are one apart and neither refers to the other; see sampling methods for how that category is left.
What you can and cannot do about it
The slide states the mechanism and stops. This is the point at which a reader most needs the material to continue, and it does not.
Working with what the source leaves you
One boundary statement belongs on every sampling page in this library, and this is where it bites hardest. The week supplies six named sampling types, three usable distinctions, this bias with its causal chain, and a cost argument for sampling. It supplies no procedure for executing any sampling type, no mention of a sampling frame, no sample size rule in its own voice, and no criterion for choosing between types. Sampling itself is named in the later data collection week's only stated learning objective and was taught there in one sentence that recommended random selection and immediately withdrew it as usually impossible. Across the six weeks now on record, fourteen objectives are stated, none fully delivered, four partially and ten not at all — audited at the six-week objective audit.
What to carry forward
- Non-response bias is the one bias the subject teaches, and its causal chain — mechanism, composition, direction — is sound and reusable.
- The chain gives you a direction of error and nothing more. No magnitude, no rate and no threshold appears anywhere in the week.
- "The most common" is an assertion with no comparison behind it, and no second bias is named on the slide to compare it with.
- Selection bias, compliance and the formal definition of a biased statistic all come from a reproduced article in another discipline. Attribute them accordingly.
- The material offers no remedy. Record approached and achieved separately, compare respondents with non-respondents where you can, and state the direction of the likely error in your limitations.
Frequently asked questions
What exactly does the supplied material say non-response bias is?
It does not define the term. It describes the mechanism: respondents choose whether to answer, and if those who decline hold different views from those who answer, a bias is introduced. The illustration then shows the estimate coming out lower than it should be. That description is the whole of the definition available.
Does random selection protect me from non-response bias?
The material says both yes and no, two slides apart, and this library does not choose between them. One slide credits random selection with avoiding the biases that arise when trying to select a representative sample; the bias slide then describes a mechanism that operates on randomly selected respondents exactly as on any others. Report both statements and design as though the more cautious one holds.
What response rate is good enough?
The teaching material names none. The week's study notes, which at that point reproduce a sport and exercise science article word for word, mention a compliance rate that journal editors in that field accept — that is a statement about one discipline's publishing conventions in 2000, not a standard for project research, and the subject sets no threshold of its own.
How is selection bias different from non-response bias here?
The two terms sit in different documents and are never related. Selection bias, in the reproduced notes, is the possibility that arises when the sample is not representative of the population. Non-response bias, in the slides, is about who answers once selected. The material never draws that distinction, so if you use both terms you should draw it yourself and say you are doing so.
Can I correct my results for non-response?
Not from this material, which describes no weighting, adjustment or non-response analysis of any kind. If you apply a correction, take the method from a text you cite and say so explicitly. What the source does support is stating the likely direction of the error and naming the mechanism you believe produced it.
References and source attribution
- The supplied teaching source: the quantitative research week's slide titled "Sampling bias", together with the sampling-types slides that precede it and the same week's study notes.
- Hopkins 2000, 'Quantitative Research Design', a sport and exercise science web journal, vol. 4, no. 1 — the article set as the week's required reading, reproduced verbatim as the study notes' sections on samples and sample size, and the source of every statement on this page attributed to the notes.
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford. Standing reference for the bias taxonomy the supplied material does not supply.
- O'Leary, Z. 2017, The Essential Guide to Doing Your Research Project, 3rd ed., Sage Publications, London.
- Veal, A. J. 2005, Business Research Methods: A Managerial Approach, Longman.
Suggested questions for Ask KEVOS
- Walk me through the non-response bias chain in the supplied material, step by step.
- Which statements about bias in this subject come from the teaching material and which from the reproduced article?
- How should I write up a low response rate in a project research report?
- Does the material say anything about fixing non-response after data collection?
- Show me every place the supplied material contradicts itself about bias.