Choosing What to Measure
Subject characteristics affect the relationship you are investigating whether you measure them or not. From the week's set reading — a published article in a different discipline — comes the case for measuring and modelling them rather than restricting your sample to avoid them.
Where this material comes from, and what that means
The three classes of thing to measure
The three classes
Characteristics of subjects
Enough information to identify the population your subjects were drawn from. In a project study: role, seniority, sector, organisation size, project type, years of experience — whatever a reader needs to know who you studied.
The independent and dependent variables
Fixed by your research question. The dependent variable is the outcome you care about; the independent variables are everything that could affect it.
Mechanism variables
Experiments only. Variables that explain how the treatment produced its effect, rather than whether it did. Rarely available in project research, and worth taking when they are.
The dependent variable is chosen by your question
The article is blunt about this and it is worth taking as blunt: once you have your research question, the dependent variable follows automatically. If your question is about schedule performance, your dependent variable is some measure of schedule performance. The design work is not choosing it — it is choosing how to measure it with as much precision as possible.
Independent variables: everything that could affect the outcome
Having fixed the dependent variable, the article says to identify all the things that could affect it. Those are your independent variables. How many of them you carry depends on how wide your study is.
HOW MANY INDEPENDENT VARIABLES TO CARRY
| Study type | The article's advice | The cost it names |
|---|---|---|
| Descriptive, wide focus | Include as many independent variables as resources allow — the interest is in estimating the effect of everything likely to matter | The more effects you look for, the more likely at least one estimate is wrong. The article names this cumulative error explicitly. |
| Descriptive, narrow focus | Still measure variables likely to be associated with the outcome — then either restrict the sample by them or include them in the analysis | You must choose between narrowing who you studied and complicating how you analyse it |
| Experimental | The timing variable and the group or treatment identity are essential and automatic | None named; these are structural rather than optional |
| Experimental, with individual differences | Either restrict the study to one subgroup, or sample across and include the variables as covariates | Loss of precision with small samples if more than a few covariates are included |
All four rows are the set reading's guidance, from a sport and exercise science context. The costs are the article's own, stated in its own terms.
The central choice: restrict or measure
This is the most useful idea in the section and it applies directly to project research. Subject characteristics affect your result whether or not you measure them. You have two ways to deal with that, and no third.
Restrict the sample
- Study only one subgroup — one sector, one project type, one seniority band
- The characteristic cannot vary, so it cannot confound
- Simpler analysis, smaller sample needed
- Your findings apply only to that subgroup
Measure and include
- Sample across the characteristic and carry it into the analysis
- The characteristic varies and is accounted for rather than removed
- More complex analysis, larger sample needed
- Your findings apply more widely — and you learn whether the characteristic mattered
Applying this to a project study
A working sequence
Write the research question as one sentence
If you cannot, the dependent variable is not yet fixed and nothing below will work.
Name the dependent variable and its measure
Not the construct — the measure. "Schedule performance" is a construct; "variance against approved baseline, in per cent" is a measure.
List everything that could plausibly affect it
Be generous at this stage. Cutting comes next, and you cannot cut what you never listed.
Decide restrict or measure, variable by variable
For each: is this something I will hold constant by who I study, or something I will record and account for?
Record enough about your subjects to identify the population
This is a reporting obligation, not an analysis one. A reader must be able to tell who your findings are about.
Check the count against your sample size
The article warns that carrying more than a few covariates with a small sample costs precision. Small samples are the normal condition in project research.
What the teaching material says about any of this
What to carry forward
- Your question fixes the dependent variable. Your design work is finding the most precise available measure of it, before you collect anything.
- List everything that could affect the outcome, then decide for each whether you will hold it constant by restricting who you study, or record it and account for it.
- Measuring and including widens where your findings apply — at the cost of a larger sample and a more complex analysis.
- Report enough about your subjects that a reader can identify the population they came from. That is a separate obligation from the analysis.
- You cannot recover a variable you did not measure. Every decision on this page is a design decision, and design closes when collection starts.
Frequently asked questions
Is this the subject's own guidance?
No. It comes from a published article on quantitative research design that the week sets as required reading, from a sport and exercise science journal. None of this section appears in the teaching material's own notes, which reproduce two other sections of the same article and omit this one.
Should I restrict my sample or measure the characteristic?
The article prefers measuring and including, because it widens where your findings apply, and it names the costs: a larger sample, a more complex analysis, and a greater chance that one of many estimated effects is wrong. With a small sample and several covariates, restricting is often the more honest choice.
What is a mechanism variable?
A variable measured in an experiment to explain how the treatment produced its effect, rather than whether it did. They are rarely available in project research, where you seldom control the treatment — but where you can measure one, it turns a finding that something worked into an account of why.
How many independent variables is too many?
The article warns that carrying more than two or three covariates with a small sample costs precision in your effect estimates, and that looking for many effects at once raises the chance at least one estimate is wrong. It gives no fixed limit, and its context is sport and exercise science rather than project research.
What if I realise mid-analysis that I needed a variable?
You cannot recover it. This is why the choice sits at design time — the article places this section before its analysis material for exactly that reason, and it is the most common defect in small project studies.
References and source attribution
- Hopkins 2000, 'Quantitative Research Design', a sport and exercise science web journal, vol. 4, issue 1, approximately 4,318 words — the Week 8 set reading. Retained as a published bibliographic citation; the journal, the reviewer and all institutional affiliations are scrubbed per the library's naming rules.
- The supplied teaching source: weekly study notes, slide decks and assessment activities for a master's-level research methods subject in project management. Author, institution and year not stated in the supplied files.
Suggested questions for Ask KEVOS
- How do I decide which variables to measure in a project study?
- Should I restrict my sample or measure the characteristic and control for it?
- What is the difference between a dependent variable and a mechanism variable?
- How many covariates can I carry with a small sample?
- What subject information do I have to report about my participants?