The Context of Data Gathering
A single slide draws the line between an instrument built before you enter the field and one built for the setting you find there. The line matters, and almost everything that follows from it is left for you to work out.
The two cases the slide contrasts
The supplied teaching source gives this subject one slide, titled "The context of data gathering". It works by contrast: what an instrument is like when it is designed in advance, and what a collection method is like when it is built for the setting.
The instrument built before the field
- Designed in advance, to accomplish a specific purpose
- "To a degree" independent of the context
- "Should" yield consistent results if used again
- Consistency across uses is a meaningful test of it
- The purpose is fixed before the setting is seen
The method built for the setting
- Developed specifically for the context the research is situated in
- Context specific by construction, not by accident
- Consistency across time and different contexts is not a meaningful test
- Rigour is offered as the substitute standard
- The purpose and the setting are settled together
The distinction is worth having and it is sharper than the usual paradigm contrast, because it turns on a question of sequence rather than of philosophy: was the instrument fixed before you saw the setting, or after? That question has an answer on every project, including projects that would not describe themselves as qualitative. It also completes an argument the material starts elsewhere, on the slide where reliability is said to have more application to quantitative than qualitative work — see Validity and Reliability.
The hedges carry more weight than the claims
Two hedges in the quantitative sentence change what it commits to, and both should survive into any use you make of it.
READING THE QUANTITATIVE CASE WITH ITS HEDGES INTACT
| As often paraphrased | As the source actually puts it | What the difference costs you |
|---|---|---|
| A designed instrument is independent of context | "to a degree independent of the context" | The source concedes partial context-dependence and never says how much, so context still has to be described in a quantitative write-up |
| The same instrument gives the same results | "should yield consistent results" | An expectation, not a guarantee. Inconsistency on a second run is not automatically an error in the instrument |
| Qualitative methods cannot be tested for consistency | "it would not be meaningful to test" | A claim about meaningfulness, not about difficulty. A later week nonetheless asks for checking, rechecking and a data audit |
The middle column is the source's own wording. The right-hand column is the practical consequence and is synthesis.
What counts as a context is never said
The whole argument rests on the word context, and the word is never defined. Nor is any list of contextual factors given. What the slide names is thin: the field, the situation the research is placed in, time, and "different contexts" as a category.
For project research the omission bites immediately. A study of a delivery process inside one organisation sits in an industry, a regulatory regime, a contract type, a delivery method, a phase of the project life cycle and a moment in that organisation's history. Any of those might be the context that makes an instrument non-portable, and the material offers no basis for saying which.
Rigour, and the two terms it is defined with
Having removed consistency as a test for qualitative work, the slide names what replaces it.
Qualitative researchers focus on establishing the rigour of their research i.e. establishing the soundness and the dependability of their research.
DEPENDABILITY IN TWO PLACES, WITH TWO MEANINGS
| Where | What it is used to mean | What it is a substitute for |
|---|---|---|
| This week, in the definition of rigour | Part of what rigour consists of, invoked immediately after consistency across contexts is ruled out | Consistency of results — it stands in for the thing the slide has just abandoned |
| The later qualitative week, defined explicitly | "the need for the researcher to account for the changing context within which research occurs and how these changes affect the way the researcher approaches the study" | Reliability, in a table pairing traditional and alternative criteria |
Accounting for change and producing the same result twice are different requirements. The material uses one word for both and never notes the difference.
Is consistency worth testing? The material answers both ways
Two further conflicts run between this slide and the later week on qualitative research. Both are recorded here and neither is resolved, because the material provides no basis for resolving them.
Two positions held at once
Not meaningful to test — but check, recheck and audit
This week says testing qualitative collection methods for consistent results would not be meaningful. The later week instructs the researcher to document the procedures for checking and rechecking the data throughout the study, and afterwards to conduct a data audit examining the collection and analysis procedures. Checking, rechecking and auditing a procedure is a consistency test on that procedure.
Built for one context — but applied across contexts
This week says qualitative collection methods are built specifically for their context, which makes them non-portable. The later week's comparison table says qualitative research "applies ideas across contexts", and the same week defines transferability as a criterion for judging qualitative work. Context-boundedness and cross-context application are asserted of the same approach in the same body of material.
There is a defensible reading in which both survive — the method is local while the ideas it produces travel — but the material never offers it, and you should not attribute it to the material. What you can do is say in your own write-up which position you are taking, and why. The wider treatment of the qualitative approach is on What Qualitative Research Is.
The design consequences the source leaves to you
The source states the contrast and stops. What follows is synthesis, built only on the distinction the slide draws — instrument fixed before the field, or method built for the setting — and offered as a way to act on it.
Working out which case you are in
Contextual facts worth recording, whichever case you are in
- The setting: what kind of organisation, function or programme the data came from, in terms that do not identify it.
- The moment: where the work sat in the project or organisational cycle when data was collected.
- The access position: who granted access, on what terms, and what that made available or closed off.
- The respondents' relation to the subject: whether they were affected by the thing being studied.
- What changed during collection: reorganisations, staff turnover, schedule shifts, anything that altered the setting mid-study.
- What in the instrument was fixed in advance and what was developed on site.
What to carry forward
- The real distinction is one of sequence: was the instrument fixed before you saw the setting, or built for it?
- Even the pre-designed instrument is only "to a degree" independent of context. The residue is yours to document.
- The source rules out consistency testing for context-built methods and puts rigour in its place, then defines rigour with two terms it does not supply.
- Context is asserted as decisive and never defined. No contextual factors are named, so the description of your setting is your own construction.
- The material says both that qualitative methods are context-bound and that qualitative research applies ideas across contexts. State your position; do not assume the material has one.
Frequently asked questions
What does the material say about instruments designed before fieldwork?
That in quantitative research the instrument is designed before the researcher goes into the field to accomplish a specific purpose, and is therefore to a degree independent of the context. If used again, the same instrument should yield consistent results. Both hedges — to a degree, and should — are the source's own.
Why does the material say consistency testing is not meaningful for qualitative methods?
Because it holds that qualitative collection methods are developed specifically for the context the research sits in. On that account, testing whether the method yields the same results over time and in different contexts is testing it against conditions it was never built for. The claim is about meaningfulness, not difficulty.
What is rigour in this material?
It is named as what qualitative researchers focus on establishing, and glossed as the soundness and the dependability of the research. Soundness is never defined anywhere in the supplied material, and dependability is defined only in a later week, with a different meaning from the one this slide implies.
Does this mean qualitative findings cannot travel to another setting?
The material does not settle it. This slide implies context-bound methods, while a later week says qualitative research applies ideas across contexts and defines transferability as a judging criterion. Both positions are in the supplied material and neither is presented as overriding the other.
What contextual factors should I record?
The material names none, so any list is constructed rather than quoted. A workable set for project research is the setting, the moment in the delivery or organisational cycle, the access position, the respondents' relation to the subject, and anything that changed during collection. Present it as your framework, not the source's.
My instrument changed once I got into the field. Which case applies?
Neither cleanly, and the material does not describe the in-between position. Treat it as a design fact rather than an embarrassment: record what was fixed in advance, what changed, when and why. An undocumented mid-study change is the version of this problem that damages a write-up.
References and source attribution
- O'Leary, Z. 2010, The essential guide to doing your research project, Sage Publications.
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford.
- Trochim, W. M. K. 2006, Research Methods Knowledge Base (http://www.socialresearchmethods.net/kb/probform.php). (The knowledge base the material points to for the qualitative criteria that include dependability.)
- Quinlan, C. 2011, Business Research Methods, 1st ed., Cengage Publishing.
- The supplied teaching source: consolidated weekly slide material and study notes on research design and on qualitative research, used here as the basis for every statement attributed to "the source", "the week" or "the material".
Suggested questions for Ask KEVOS
- Write a context statement for my study that another site could compare itself against.
- Was my instrument fixed before fieldwork or built for the setting, and what follows for how I defend it?
- Which features of my research setting would have to change before my instrument stopped working?
- Draft the paragraph explaining why I adjusted my interview schedule partway through collection.
- What quality criteria should I claim for a single-site qualitative study, and where do I cite them from?