Rigour, Triangulation and Validity in Practice
Rigour is not a claim you make in a sentence. It is a set of arrangements a reader can inspect. This page takes apart the arrangements one examined doctoral thesis actually put in place, and marks the point at which its evidence stops being about research design at all.
The rigour problem this thesis had to solve
The work in question is an insider action research doctorate in an engineering organisation. The researcher was a manager in the organisation being changed. That is the maximum-access, maximum-entanglement position: nobody is better placed to see what happened, and nobody has more at stake in the account of it.
The entanglement goes further than access. The researcher's own leadership was measured before and after with a standardised inventory and reported as a finding, which makes the researcher a unit of analysis in his own study. The thesis is written entirely in the first person, a choice defended in the opening chapter by citing three earlier action research doctorates as precedent.
Every figure on this page is a finding of that single study. None is a threshold, a target or a benchmark for anybody else's design. What is transferable is the shape of the arrangements, not their size.
Three of four triangulation types, and the one not claimed
The thesis names four types of triangulation by reference to a cited methodological author, and claims three of them. Theory triangulation is named and is not claimed.
TRIANGULATION AS CLAIMED IN ONE EXAMINED THESIS
| Type named | Claimed? | Defined in the supplied material? | What the structural record shows in the study |
|---|---|---|---|
| Data | Yes | Not defined in source | 17 named data sources gathered over a field period of three and a half years, with the participant base expanding across phases from 6 to 26 to 60 to 108, plus 50 project team members on the customer side |
| Methodological | Yes | Not defined in source | Qualitative content analysis with coding and categorisation, then cross-case analysis, alongside a deliberately limited quantitative strand of frequencies and means |
| Investigator | Yes | Not defined in source | A second investigator, identified in the thesis, who co-developed the coding |
| Theory | No | Not defined in source | Nothing. The type is named in the thesis and expressly not claimed |
Column four reports what the structural record of the study contains. It is not the thesis's own mapping of evidence to type, and it is not a definition of any type.
The instructive part is the refusal. It would have cost nothing rhetorically to write "triangulation was achieved" and leave the reader to assume all four. Naming four and claiming three tells a reader exactly where the study stopped, and converts a vague assurance into a checkable statement. That is the same discipline that separates a bounded contribution claim from an unbounded one — see Stating Limitations and Contribution.
The second investigator, and what co-developed coding buys
Investigator triangulation in this study takes a specific form: a second investigator co-developed the coding. It is worth being precise about what that is, because two different controls are commonly described in the same words.
What co-development gives you
- The category set is not one person's reading of the data
- Ambiguous categories get argued about before they are applied, when they can still be changed
- The scheme has to be explicit enough for a second person to use, which forces definitions onto paper
- It is a control on the instrument of analysis
What it does not give you
- It is not independent double-coding of the same material after the scheme is fixed
- It produces no agreement statistic, and the structural record of this study shows none
- It does not test whether a third person would apply the scheme the same way
- It is a control before coding, not a measurement of coding
Neither is superior in the abstract; they answer different questions. What matters is that your write-up says which one you did. Across the examined set the postures differ sharply: this thesis co-developed its coding with a second investigator; the mixed methods thesis analysed ten interviews manually and stated plainly that with only ten, coding software was not needed; the conference paper reports no inter-coder check of any kind against a corpus of 2,015 articles.
One standardised instrument, and everything else declared
The study used one standardised published instrument, administered before and after. Every other instrument was researcher-built, and the thesis states expressly that those instruments are not standardised.
That combination does two things at once. The standardised instrument supplies a measure the researcher did not design and cannot tune, which allows a before-and-after comparison that does not depend on the researcher's judgement — including, in this study, on the researcher's own leadership. The explicit non-standardised declaration then prevents the credibility of the published instrument from bleeding across onto the home-made ones.
TWO KINDS OF INSTRUMENT IN ONE STUDY
| The standardised published instrument | The researcher-built instruments | |
|---|---|---|
| How used | Administered before and after the intervention | Built for specific phases of the fieldwork |
| What it supports | A comparison over time on a measure external to the study | Coverage of constructs no published instrument addressed |
| How declared | Named as standardised and published | Expressly stated to be non-standardised |
| Inspectable by a reader? | By obtaining the published instrument | Yes — the instruments and their scored results are reproduced in the appendices |
A single study's arrangement, not a template. See Appendices and Research Instruments for what that appendix actually contained.
Refusing the statistics the sample could not support
The quantitative data in this thesis is deliberately limited to frequencies and means. Higher-order statistics were refused, and the refusal is stated with two grounds: the sample was too small, and further statistical work would have added little to rich qualitative data.
Two grounds, not one, is the detail worth copying. The first is a constraint. The second is a design argument — it says the additional analysis would not have earned its place even if the sample had allowed it. A reader can disagree with either, which is what makes the refusal an argument rather than an apology.
Set that against the other two empirical postures in the examined set, all of them findings of the individual studies concerned.
Three postures towards statistical analysis
Frequencies and means only
The action research thesis limits itself to frequencies and means and states why it goes no further. The qualitative strand carries the analytical weight, so the numbers are supporting evidence rather than the argument.
ANOVA and regression on 71 valid responses
The mixed methods thesis runs one-way ANOVA with post-hoc tests and multiple regression, and defends its sample adequacy by citing two published participants-per-predictor rules of thumb. The defence is offered before the analysis, not after.
Mean, median, mode and standard deviation
The portfolio thesis reports descriptive statistics with no inferential testing at all, and where two means are close it says outright that they are too close to conclude anything — then contrasts that with a gap it does treat as meaningful.
Deciding how far to take the numbers
External signals, and what they are actually worth
Alongside the design controls, the study points to signals from outside the research process: the changed way of working was accepted by the customer, and the work attracted an internal organisational award. Both are offered as indications that the findings hold beyond the study itself.
Take them seriously and take them for what they are. In applied research, uptake is genuine evidence that something worked in its setting, and it is evidence a purely academic study cannot produce.
One further absence is worth registering, because it shapes how this thesis argues for itself. It does not declare a paradigm in ontology and epistemology terms at all; its methodology is named as action research and described as a meta-methodology and as emergent. Rigour here is demonstrated procedurally rather than philosophically — see Declaring a Research Paradigm, Or Not.
Assembling a rigour statement a reader can inspect
The supplied teaching source prescribes nothing about rigour. The checklist below is synthesis, assembled from what this study did and from the contrasts the other four examined works supply. Each item is written so that it either exists as a document or it does not.
What to be able to point at
- Every data source, listed and counted, rather than described as multiple
- The period over which data was collected, and how the participant base changed across it
- Any triangulation type you claim, named, defined from the text you are citing, and matched to the specific evidence that supports it
- Any triangulation type you considered and did not achieve, named as not claimed
- Who else touched the coding, at what stage, and whether they co-developed the scheme or applied it independently
- Which instruments are published and standardised, and which you built — stated in those words
- What your researcher-built instruments were derived from
- The instruments themselves, reproduced in an appendix, with their scored results
- Your analytical ceiling, and the reasons you stopped there
- Any external signal of uptake, filed under findings rather than under validity
What to carry forward
- Name the triangulation types you claim and the ones you do not. Three of four, stated, is stronger than an unqualified assurance.
- Never claim a type you cannot define from the text you are citing — the examined work's own source names four and defines none of them for you.
- Say which coding control you used. Co-developing a scheme with a second investigator and independently double-coding fixed material are different arrangements with different guarantees.
- Declare researcher-built instruments as non-standardised, say what they were derived from, and reproduce them in an appendix.
- State the ceiling on your analysis and the reasons for it. Two reasons — a constraint and a design argument — is the pattern worth copying.
- Uptake, acceptance and awards belong in findings. They are not controls on your research process, and in insider research they are the easiest thing to mistake for one.
Frequently asked questions
How many triangulation types do I need to claim?
The supplied material sets no number, and the examined thesis claimed three of the four its cited author names. What matters is that each claimed type is matched to specific evidence in your study and that any type you did not achieve is named as not claimed. A single well-evidenced type is worth more than four asserted ones.
Is a second coder essential for qualitative analysis?
The supplied source does not require one, and practice across the examined works varies: one thesis had a second investigator co-develop its coding, another analysed ten interviews manually and said outright that software was unnecessary at that scale, and a conference paper coding 2,015 articles reports no inter-coder check at all. Decide by the size and contestability of your coding task, and state what you did.
Can I use an instrument I designed myself?
Yes, and most applied project research has to, because the constructs are local. Declare it as researcher-built and non-standardised, say what it was derived from, and reproduce it in an appendix so a reader can judge it. The examined thesis did exactly this alongside one standardised published instrument.
Why would a researcher deliberately refuse to run more statistics?
Because the analysis has to be earned. The examined thesis gave two reasons — the sample was too small, and the additional work would have added little to rich qualitative data. Stating a ceiling with reasons is an argument a reader can engage with; running a test the data cannot support invites a reviewer to reject the whole quantitative strand.
Does my organisation adopting my recommendations prove my findings are valid?
No. Adoption is evidence that the intervention was acceptable in its setting, which is worth reporting as a finding. It says nothing about whether your account of how and why it worked is accurate, and it is not a control on sampling, coding or analysis. Keep uptake and validity in separate sections.
I am researching my own workplace. What does that change?
It raises the access and the entanglement together. The examined thesis handled it structurally rather than by disclaimer: a standing heading in every case study for the researcher's own role, a standing heading for difficulties encountered, a second investigator on the coding, and one external standardised measure the researcher could not tune. Design the controls in rather than declaring the problem away.
References and source attribution
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford.
- O'Leary, Z. 2017, The Essential Guide to Doing Your Research Project, 3rd ed., Sage Publications, London.
- Quinlan, C. 2011, Business Research Methods, 1st ed., Cengage Publishing.
- An examined doctoral action research thesis in an engineering organisation, read structurally from its front matter, contents, methodology chapter and appendices. Every count and figure attributed to it here is a finding of that study alone. Four triangulation types are named within it by reference to a cited methodological author; none is defined in the material supplied to this library.
- The supplied teaching source: consolidated weekly teaching notes and slide material, which contains no treatment of triangulation, validity or rigour.
Suggested questions for Ask KEVOS
- Draft a rigour statement for a small qualitative study of my own project portfolio.
- What evidence would I need before I could claim investigator triangulation?
- My sample is 24 responses. Which analyses can I defend and which should I refuse?
- How do I declare a questionnaire I designed myself without undermining it?
- I am researching my own team. What structural controls can I build into the write-up?