The Wording of Questions
This is the strongest slide in the week and the one worth working through properly. Six checks, plus two more carried over from the design slides, with each fault set out as a question that fails and the same question repaired.
The wording slide, in full
Wording is the one part of this week that repays close reading. The checks are specific, they are the kind a practitioner can actually run over a draft, and there are enough of them to catch most of what goes wrong in a first draft of an instrument.
The opening instruction is the week's strongest requirement and its least specified one — rehearsal is required, and no procedure, number of rehearsers or success criterion is given for it. That gap, and what a minimum rehearsal might look like, is treated on Designing a Questionnaire.
One piece of context, stated once. The week's single learning objective is to evaluate methods of data collection, and the delivery audit records that verb as undelivered — the week compares no two methods on any criterion. Wording is the exception in quality, not in kind: it tells you how to write a question well, and nothing about which instrument to choose.
The faults the material names
Two more faults sit on the design slides next door: questions must be "not leading i.e. one suggesting the answer", and must not be "misleading". Set out together, the material names eight things to check.
THE EIGHT CHECKS, WHERE EACH IS STATED, AND WHAT EACH ONE CATCHES
| Fault | Where the material states it | What goes wrong when it is present |
|---|---|---|
| Leading | Design point 5 — "not leading i.e. one suggesting the answer" | The question tells the respondent which answer is expected, so the answers measure the question rather than the respondent |
| Confusing or misleading | Wording check 1; reminder 3 | Different respondents answer different questions, and you cannot tell which |
| Misinterpretable by a non-English speaker | Wording check 1 | A subset of your respondents answers a question you did not ask |
| Assuming | Wording check 2 — including "a particular cultural understanding" | Respondents for whom the assumption is false have no honest answer available |
| No time-frame | Wording check 3 | Answers refer to different periods and cannot be compared or aggregated |
| Insensitive, too direct, too personal | Wording check 4; design point 4 | Non-response on that item, or a polite answer instead of a true one |
| Jargon or acronyms | Wording check 5; reminder 4 | Respondents guess at the term, and their guesses differ |
| Imprecise | Wording check 6 — "Do the words define what you want to know?" | You collect an answer to a vaguer question than the one your research question needs |
Six of the eight come from the wording slide. Leading comes from the design points and misleading is stated in two places under two names.
Worked repairs
The material names every fault and repairs none of them. The repairs below are this page's constructions, written in a project delivery setting so that the mechanics are visible. The faults are the source's; the before-and-after pairs are not.
SIX FAULTY QUESTIONS AND THE SAME QUESTIONS REPAIRED
| Before | What fails | After |
|---|---|---|
| "Don't you agree that the new gate review process has improved delivery?" | Leading. The wording supplies the expected answer, and disagreeing now costs the respondent something | "What effect, if any, has the gate review process had on delivery in the projects you work on?" |
| "How often does your project board review the risk register?" | Assuming. It takes for granted that there is a board, that there is a register, and that reviews occur | "Does your project maintain a risk register?" followed, for those who answer yes, by "How often is it reviewed?" |
| "Is the RAID log kept up to date under the current framework?" | Jargon and acronyms. The term and the framework may mean different things to different respondents, or nothing at all | "Is a log of risks, assumptions, issues and dependencies kept for your project, and is it updated at least monthly?" — with the two halves asked separately |
| "How many change requests do you raise?" | No time-frame. One respondent answers for the current project, another for the year, another for their career | "In the last six months, how many change requests did you raise?" — six months is this example's value, not a figure from the source |
| "Was the project successful?" | Imprecise. The words do not define what you want to know, so every respondent supplies their own definition of success | "Which of the following did the project meet: the agreed scope, the approved budget, the approved schedule? Select all that apply." |
| "What is your salary?" | Too direct and too personal, which the design slide prohibits outright | A banded, clearly optional item, asked late in the instrument. The banding technique is this page's; the material prohibits the fault and supplies no remedy |
Each repair does one of three things: it removes the pressure towards an answer, it splits an assumption out into its own question, or it replaces a word that respondents would define for themselves.
The pattern underneath the repairs
Five of the six repairs are the same move made in different places: the question was carrying something it should have asked about. A leading question carries your expected answer; an assuming question carries a fact about the respondent; a vague question carries a definition. Taking that thing out of the question and asking it separately is the whole technique, and it is why repaired instruments almost always have more questions than faulty ones — which is exactly what design point 1 recommends when it prefers many short questions to a few long ones. Each repaired question still needs its response options settled alongside the wording; the formats the material offers are on Levels of Measurement in Structured Questions.
The deck's own worked example, repaired
Check 3 asks whether the question determines a time-frame, and the material supplies an example to show how it is done: "do you think the Federal Government will change superannuation legislation in the next year?" The time-frame is there, and the phrase "in the next year" is the only time-frame stated anywhere in the deck's question guidance.
What the example does well
- It fixes the period the answer refers to — "in the next year"
- Without it, one respondent answers about the next month and another about the next decade
- Adding a period is the cheapest single improvement you can make to a draft question
What a repair would have to change
- Stop asking for a prediction about a third party's future conduct, which no respondent can answer as fact
- Ask instead about the respondent's own position or experience within the period
- This page's version: "In the next twelve months, is your team planning any change to how it handles this? Yes / No / Not yet decided" — the structure is this page's, not the source's
The fault the material never names
It is worth being explicit about why this belongs on the page. The value of a wording checklist is that it can be run mechanically over a draft. A checklist that is missing a common fault will pass questions that should not pass, and the reader is better off knowing which fault is absent than assuming the eight checks are complete. The same caution applies to the week's one comparison of question types on Structured and Unstructured Questions: what is listed is not all there is.
Checks the material states and cannot answer
Three of the six wording checks raise a constraint and attach no remedy to it. They are worth running anyway — knowing a question will fail for a group of your respondents is better than not knowing — but the fix has to come from elsewhere.
- Non-English speakers. Check 1 asks whether a question can be misinterpreted by a non-English language speaker. No translation, back-translation or review procedure is named anywhere in the week.
- Cultural assumptions. Check 2 asks whether the question assumes a particular cultural understanding. No review process, no adviser role and no worked example of a cultural assumption is supplied.
- Jargon and acronyms. Check 5 bans them and no glossary practice, expansion convention or plain-language substitute list is offered.
- Reading level. The design slide asks whether your population has good reading skills; nothing follows it there either.
Running a wording review over a draft
The source supplies the checks and no procedure for applying them. The sequence below is this page's, and it is built so that each pass looks for one thing — reviewing for eight faults at once is how faults survive review. For the instrument as a whole, rather than its questions one at a time, see Designing a Data Collection Instrument: A Worked Attempt.
Four passes over a draft instrument
Pass 1 — read every question as a hostile respondent
For each question, invent a respondent for whom it is unanswerable: no risk register, a different definition of success, a role the question does not fit. Every one you find is an assumption to split out into its own question.
Pass 2 — underline every word the respondent has to define
Effective, regularly, successful, senior, major, timely. Each underlined word either gets a definition in the question or gets replaced by the thing you actually want to know.
Pass 3 — check the period and the pressure
Does each question state the period it refers to? And does any question tell the respondent which answer you are hoping for? Those are checks 3 and the leading-question rule, and they are the two fastest to run.
Pass 4 — read it aloud to someone who is not on the project
This is the source's own instruction, and the only remedy it supplies for any of the faults. Ask them to say what each question is asking before they answer it. The gap between their reading and your intention is the fault the earlier passes missed.
Wording checklist — the eight faults, in the form of a pass or fail
- No question suggests the answer you expect
- No question can be read two ways by two reasonable respondents
- No question assumes a fact about the respondent that you have not established with an earlier question
- Every question that refers to a period states the period
- No question asks two things at once — the check the material does not supply
- No jargon, no acronyms, no internal shorthand
- Sensitive items are placed late, worded plainly, and marked optional
- Every question's words define what you actually want to know, in terms you could defend to a reader of the finished report
- The whole instrument has been read aloud to someone outside the project, with the changes recorded
What to carry forward
- The wording slide is the best-developed content in the week: six checks, specific enough to run over a draft.
- Two further faults sit on the design slides — leading questions, and misleading ones — and the material never distinguishes the two words.
- Every check is stated as a question to ask yourself, and not one comes with a repair. The repairs on this page are constructed here.
- Most repairs are the same move: take out what the question was carrying — an expected answer, an assumed fact, an undefined word — and ask it separately.
- The slide's own example of good practice asks a respondent to predict a third party's conduct, which the design slide's rule against leading questions prohibits.
- The material never names the fault of asking two things in one question. Add that check yourself.
- Three checks raise constraints — non-English speakers, cultural assumptions, jargon — and no remedy for any of them appears in the week.
Frequently asked questions
What wording faults does this material actually name?
Eight, across three slides: leading, confusing or misleading, misinterpretable by a non-English speaker, assuming, missing a time-frame, insensitive or too direct or too personal, jargon-laden, and imprecise. Six come from the wording slide; leading and misleading come from the design points and reminders beside it.
Does the material show how to fix a badly worded question?
No. Every fault is stated as a check to run, and no repaired version of any question appears anywhere in the week. The before-and-after pairs on this page are constructed for the purpose; the faults they demonstrate are the source's.
Is a question that asks two things at once covered?
No, and it is worth knowing that before you rely on the checklist. The nearest thing in the material is the design point preferring many short questions to a few long ones, which is a statement about length rather than a named fault. Add the check yourself: any question containing "and" or "or" between two different things should be split.
How do I word a question about something sensitive?
The material gives you three incompatible positions and no technique. One slide prohibits questions that are too personal, one asks you to consider sensitivities such as age and income, and the wording check asks whether the wording is sensitive and also whether it is too personal. Take the technique from your ethics approval process and from published instruments, and place sensitive items late and optionally.
What is wrong with the deck's own example question?
It demonstrates a time-frame correctly and, at the same time, asks the respondent to predict a third party's future conduct and to say what they think will happen — which is the kind of steer the design slide's rule against leading questions prohibits. The material offers it as good practice and never notices the conflict.
How many people should review my wording?
The material says the wording must be "rehearsed with someone else before you begin" and calls that essential, without stating a number, a procedure or a criterion for success. Treat one reader as the floor rather than the target, and record what changed after the review so that the rehearsal leaves evidence behind.
References and source attribution
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford.
- O'Leary, Z. 2017, The Essential Guide to Doing Your Research Project, 3rd ed., Sage Publications, London.
- Veal, A. J. 2005, Business Research Methods: A Managerial Approach, Longman.
- Naoum, S. G. 2013, Dissertation Research & Writing for Construction Students, 3rd ed., Routledge.
- The supplied teaching source: the week 10 slide deck on collecting data, the slide on the wording of questions, read with the two questionnaire design slides that precede it.
Suggested questions for Ask KEVOS
- Run the eight wording checks over my draft questions and show me the failures.
- Repair this question for me and tell me which fault it had.
- Which of my questions assume something about the respondent that I have not asked about?
- Add a time-frame to each of my questions and tell me where the period changes the meaning.
- Which words in my instrument would a respondent have to define for themselves?