Designing a Data Collection Instrument: A Worked Attempt
Six hundred words were set; 590 came back, carrying an approach, a borrowed sixteen-method list and no instrument. The teaching material never displays an instrument either, so this page reads the two together and sets out what the guidance that was available would have produced.
The task, and what came back
Read the sentence structurally and the trouble is visible before any student touches it. Two tasks are joined by a conjunction and share one word count with no division between them, and the second - discuss why this is suitable - can be answered completely without doing the first.
The submission answering it takes a topic of the writer's own: the success and failure of research and development projects. It runs to roughly 590 words of body text against the 600 set - a count derived for this library, not declared in the piece.
WHAT THE TASK ASKED FOR AND WHAT ARRIVED
| Asked for | Delivered | Reading |
|---|---|---|
| A data collection tool, specific to the study | No instrument of any kind | The primary instruction, and the one not carried out |
| Discussion of why it suits the purpose | A stated preference for stakeholder interviews, with reasons | The secondary instruction, and the one carried out |
| 600 words | About 590 words of body text | Roughly 1.7% short; a derived count, not a declared one |
| Specific to the writer's own study | An R&D success-and-failure topic sustained throughout | Held consistently from first line to last |
The material never displays an instrument
Before reading the submission as a failure of effort, look at what it had to work from. Across the four weeks of supplied material - research design, qualitative research, data collection and quantitative analysis - no questionnaire, interview schedule, observation protocol, focus group guide or coding sheet is displayed anywhere.
What the submission does deliver
It is the most substantial of the three submissions supplied with this material and the only one organised into distinct functional sections. Several of its moves are better than the material that set it.
Six things the piece gets right
It chooses, and gives reasons
Stakeholder interviews are named as the preferred starting point, on the stated grounds that they extract data addressing project impact and make the results easier to analyse and compare. That is precisely the discuss-why half of the brief.
It separates primary from secondary tools
Online surveys, existing performance records and the combining of statistical results are set out as secondary to direct interviews. The material names primary data in a slide heading and never develops the distinction at all.
It brings a method-and-purpose list
Sixteen methods, each with a stated purpose, six of them interview variants. It is the fullest method-and-purpose inventory anywhere in the supplied documents - and it came from an external 2018 source, not from the subject.
It reaches forward into coding
The piece states an a-priori coding scheme driven from the research questions downward, which matches the deductive timing the following week teaches. It was written before that week was taught.
It raises standardisation across collectors
A multi-member team drawn from different places and cultures will approach collection differently, so the tool must standardise across them. The material never raises this, and for organisational research it matters a great deal.
It states an end goal
One closing sentence ties the instrument back to the research aim - understanding R&D success and failure by analysing patterns and trends. Many longer pieces never do this.
The guidance that was available, and what was used
The material is thin on instruments and it is not empty. Six slides of the same week carry specific, usable construction guidance. The table sets each item against whether the submission engaged it, because the honest account of this piece is neither that it ignored the guidance nor that it followed it.
SOURCE GUIDANCE AVAILABLE FOR THIS TASK, AND ITS USE IN THE SUBMISSION
| What the source supplies | Where it sits | Engaged in the submission? |
|---|---|---|
| Four survey construction activities: formulating questions and response categories; writing background information and instruction; working through organisation and length; determining layout and design | Survey construction slide | No. None of the four is performed. No question, no instruction text, no length decision, no layout. |
| Nine questions to ask yourself before designing: content and purpose, response format, the population's knowledge, its reading skills, wording, facilities and timeframes, complexity, sensitivities, and comparable methods | Designing a questionnaire (1) | Partly. Content and purpose, complexity and comparable methods are all addressed. Response format, population knowledge, reading skills, timeframes and sensitivities are not. |
| Six design points: short and simple; unambiguous; as short as possible; neither irrelevant nor too personal; not leading; in logical sequence | Designing a questionnaire (2) | Cannot be checked. With no question written out, none of the six has anything to apply to. |
| Five reminders: sequence appropriately; clear layout; avoid misleading questions; use familiar language; thank the respondent | The Remember to slide | No, with one partial exception. The standardisation paragraph approaches the familiar-language concern from the collectors' side rather than the respondents'. |
| Six wording checks, including ambiguity, cultural assumptions, time frames, sensitivity and jargon | Wording of questions slide | No. Wording is never addressed, because no wording exists. |
| Response formats: dichotomous, and the nominal, ordinal and interval levels with a worked coding example for two of them | Choices of responses slide | No. No response format is selected, described or shown. |
| Four interview types: standard-stimulus, topic-only guide, conversational and facilitated group | Types of interviews slide | Partly. Interviews are chosen as the primary method, but the six interview variants named come from the external list, not from the four the material supplies. |
| Rehearse the wording with someone else before you begin - the material's entire pre-testing content, and it is not called pre-testing | Wording of questions slide | No. No rehearsal, pilot or trial is mentioned. The words pilot, pre-test and trial appear nowhere in the week either. |
Each item in the first column is guidance the material genuinely supplies. The full treatment of each is on the questionnaire design, question wording, levels of measurement and interview types pages.
Read down the third column and the pattern is clear. Everything the submission engaged is upstream of the instrument - what to collect, from whom, why. Everything it passed over is the instrument itself: format, wording, sequence, layout, testing. That is exactly the boundary the teaching material also stops at.
From that guidance to an actual instrument
Eight moves that turn the available guidance into a tool
List what each question is for
The material's first step in data collection is to define the variables for which you want to reach conclusions. Write the list before you write any question, and put each question against one of them. A question that serves nothing on the list does not go in.
Choose the response format first
The second of the nine design questions asks what response format will collect the best information. Settle dichotomous, ranked or scaled before you draft: the format governs the wording and what you can do with the answers afterwards. See Levels of Measurement in Structured Questions.
Draft the questions against the six points
Short and simple, unambiguous, not leading, neither irrelevant nor too personal, in a logical sequence. Two of the six are the same requirement stated twice, so there are five distinct tests. See Designing a Questionnaire.
Run the wording checks
Six of them, covering ambiguity, cultural assumptions, time frames, sensitivity and jargon. They are checks without remedies attached, so where one fails you are on your own for the fix. See The Wording of Questions.
Sequence and lay it out
Logical sequence is a design point; clear layout is a reminder; determining layout and design is a construction activity. All three say the same thing from different slides and none says how. Decide the order deliberately and be able to defend it.
Write the front matter
Background information and instruction is one of the four construction activities, and the interview preparation list separately requires written information on the project background and purpose to be sent in advance. This is the part most drafts omit and the part respondents actually read first.
Rehearse it with someone else
The material calls this essential and specifies nothing about it - no number of rehearsers, no procedure, no criterion for success, no revision loop. Do it anyway, and record what changed as a result, because that record is the only evidence of testing your method section will have.
Attach the ethics wrapper
Consent, confidentiality, anonymity, risk disclosure, data access and data protection are all required by the interview conduct material and none of them is defined there. Bring the definitions from elsewhere and cite them. See Conducting, Recording and Transcribing Interviews.
The one specification the submission does state
Two further borrowings sit nearby: a seven-term framework of four process-performance drivers and three result categories, quoted with no source given, and a compound coding label the teaching material never supplies. The second is the telling one. The following week states an objective of evaluating coding options and then names none, and the student named one from outside the subject a week before it was taught.
Citations, and a borrowed block
Three references are listed and there is one in-text citation. The citation does not correspond to any of the three: it uses a bracketed numeral against a list that is not numbered, and names a 2018 author who does not appear among three 2020 entries. The single largest piece of borrowed content in the piece - the sixteen-method list - is therefore untraceable.
The list is also introduced with an attribution line and then set out as running prose without quotation marks, with the first item shifted into the first person. In the writing as it stands, a reader cannot see where the borrowed material starts and stops.
Before you submit an instrument
Pre-flight on a design-a-tool task
- There is an artefact - a numbered list of questions, a schedule, a form or a protocol - that a reader could pick up and administer without you in the room
- Every question on it maps to something on your list of what you need to reach conclusions about
- The response format is stated for each question, not implied
- The front matter exists: what the study is, who you are, what happens to the answers, and how long it will take
- The wording has been through the six checks, and you can say what changed as a result of rehearsing it
- The consent, confidentiality and data-access statements are attached to the instrument rather than left in the method chapter
- The word budget is split deliberately between the instrument and the rationale, because the task sets both and divides neither
- Where the instrument itself is long, it goes in an appendix and the rationale carries the argument - see Appendices and Research Instruments
What to carry forward
- Design a tool and discuss its suitability are two tasks. If a brief joins them under one word count, split the budget yourself and say how you split it.
- This submission answered the discussion half well and produced no instrument - and the material it was set from displays no instrument anywhere across four weeks.
- The guidance that does exist is upstream-heavy: what to collect and from whom. Format, wording, sequence, layout and testing are where both the submission and the material stop.
- An instrument is finished when someone else could administer it without you. Until then you have an approach, not a tool.
- Any number you set for your own instrument - questions per variable, respondents, duration - is your rule and should be stated as yours.
Frequently asked questions
What counts as a data collection tool?
On the evidence of this material, nothing does: no questionnaire, interview schedule, observation protocol, focus group guide or coding sheet appears anywhere in the four weeks supplied. A workable test is administrability - a reader should be able to pick the artefact up and collect data with it without you present. An approach, however well argued, is not a tool.
Was the submission wrong to discuss rather than design?
It answered one of the two instructions in the brief and not the other, and the one it left out was the primary one. But the material it was working from never displays an instrument, and the only place an interview schedule is named is an optional video with no usable address. The shortfall is shared, and reading it purely as a lapse of effort misses what the record actually shows.
How many questions should my instrument have?
The supplied material states no number of questions, no completion time and no page limit. Its only reference to length is an instruction to work through organisation and length. The submission set itself a rule of at least two questions per variable, which is that piece of work's own specification and not a standard you can cite.
Do I have to pilot my instrument?
The material never uses the words pilot, pre-test or trial. Its entire pre-testing content is one instruction, prefaced as essential, to rehearse your wording with someone else before you begin - with no procedure and no criterion attached. Do it, record what changed, and take the piloting procedure itself from a second source.
Where should the instrument go in my report?
Conventionally the instrument goes in an appendix and the body carries the rationale, which also solves the word-budget problem when a brief sets both under one count. The published-practice treatment is on the appendices and research instruments page. The teaching material states nothing about where an instrument belongs.
Can I use the sixteen-method list the submission quotes?
Not from this material, and not from the submission. The list is introduced with an attribution to a 2018 author who appears in no reference list, and it is reproduced without quotation marks. If you want a method-and-purpose inventory of that kind, find the original, read it and cite it yourself.
References and source attribution
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford.
- O'Leary, Z. 2017, The Essential Guide to Doing Your Research Project, 3rd ed., Sage Publications, London.
- Naoum, S. G. 2013, Dissertation Research & Writing for Construction Students, 3rd ed., Routledge.
- Veal, A. J. 2005, Business Research Methods: A Managerial Approach, Longman.
- The supplied teaching source: the week 10 slide deck on collecting data - the activity slide setting the 600-word design task, and the slides on survey construction, questionnaire design, question wording, response choices and interview types from which the available guidance is drawn.
- A student submission answering that task, supplied with the teaching material and used here anonymised, with no identifying detail of the writer, the study site or the institution.
Suggested questions for Ask KEVOS
- Turn my research questions into a draft interview schedule I can put in an appendix.
- Which parts of this teaching material actually help me build an instrument, and which do not?
- Help me split a 600-word budget between an instrument and its justification.
- Draft the front matter and consent statement that goes at the top of my questionnaire.
- What should I record about rehearsing my instrument so it counts as evidence of testing?