Data Collection Methods: Data as Evidence
This was the thinnest part of the supplied teaching source: one paragraph of principle and thirteen method names with nothing attached to them. Three further weeks have since narrowed the gap substantially. This page states what is now taught, what is still only named, and where each of the thirteen labels stands.
Data constitutes evidence, and that is the whole of the principle
Thin as it is, the principle is not trivial, and it is worth reading as an instruction rather than an observation. If data is evidence, then every collection decision is an evidentiary decision: what will this establish, and will it hold up when someone contests it?
That framing rules out a common habit. Collecting data because it is available, or because a survey is easy to distribute, produces material that is not evidence for anything in particular. Be able to say which of the two things you are doing — exploring a phenomenon, or establishing a case — before you collect.
Thirteen methods, none of them defined
The apex of the methodological pyramid carries thirteen data collection methods, six in a left-hand column and seven in a right-hand column. They are names on a diagram. The supplied source attaches no definition, no example, no guidance on when to use each, and no reading to any of them.
They are set out below with a family grouping, added here to make the list navigable. The grouping is an organising convenience introduced on this page and is not source content. The figure itself is explained in The Methodological Pyramid.
ALL THIRTEEN DATA COLLECTION METHODS ON THE APEX TIER
| Method (as labelled on the figure) | Family (grouping added here) | Status in the supplied source |
|---|---|---|
| Questionnaires | Asking people directly | Not defined in source |
| Interviews | Asking people directly | Not defined in source |
| Focus groups | Asking people directly | Not defined in source |
| Scales | Asking people directly | Not defined in source |
| Observation | Watching and recording | Not defined in source |
| Projective techniques | Eliciting indirect responses | Not defined in source |
| Vignettes | Eliciting indirect responses | Not defined in source |
| Life history | First-person accounts | Not defined in source |
| Narrative | First-person accounts | Not defined in source |
| Diaries | First-person accounts | Not defined in source |
| Documents / Records / Archives | Existing material | Not defined in source |
| Internet research | Existing material | Not defined in source |
| Images | Visual material | Not defined in source |
Left-hand column on the figure: observation, questionnaires, interviews, focus groups, scales, projective techniques. Right-hand column: images, vignettes, life history, narrative, diaries, documents/records/archives, internet research. The figure attaches no meaning to the split between columns.
What the later weeks added
Three further weeks of the subject — research design, gathering your data, and quantitative analysis — supply the treatment this page could not when it was first written. The thirteen-page data collection stream is the result.
FROM A BARE LABEL TO A TREATMENT
| Named on the pyramid | Now taught? | Where |
|---|---|---|
| Questionnaire | Yes, in detail | Questionnaire design, structured and unstructured questions, question wording, levels of measurement |
| Survey | Yes, in part | Survey research and survey construction and questionnaires and response rates |
| Structured interviewing | Yes | Interview types and choosing between them and conducting, recording and transcribing interviews |
| Unstructured interviewing | Yes | Same two pages, plus the qualitative treatment at qualitative methods |
| Case study | Named and described | Qualitative methods: the commonly used set |
| Participant observation | Described, not operationalised | Same page. No observation schedule or field-note format is shown anywhere. |
| Focus group | No — one sentence | Named as recommended in the data-gathering week and never described |
| Sampling (implied throughout) | No | Sampling and selecting participants records exactly how little is supplied |
| Content analysis · documentary analysis | Partly | Documentation is one of the qualitative methods described; content analysis is not |
| The remaining pyramid labels | No | Named on the diagram and treated nowhere under that name |
“Now taught” means the activity is described under some name in the supplied material. It does not mean the pyramid's own label is defined.
What the source does define about data
The absence of method definitions makes it easy to overlook the definitions that are present. Four are directly useful when you specify a collection approach.
Definitions the supplied source provides
- Methods
- "The way in which the data to be used in the study is gathered." A second table in the same source glosses method more broadly as "ways of gathering and analysing data", with techniques as an associated term.
- Primary data
- "Data directly observed or gathered by the researcher engaged in the research project. These sources are relevant, timely and reliable."
- Secondary data
- "Use of existing data gathered by other people/organisations for other purposes. These sources are interpreted by others."
- Quantitative collection
- The source states that quantitative data collection methods "rely on random sampling and structured data collection instruments that fit diverse experiences into predetermined categories", and that these methods produce results that can be easily summarised, compared and generalised.
The primary and secondary distinction does real work in a project setting, where a great deal of usable material already exists in registers, reports and records. The source's warning about secondary sources is that they were gathered for other purposes and have been interpreted by others; see Primary and Secondary Data Sources.
The quantitative statement is the one place the source describes how a collection method actually works, and it names a trade-off worth noticing: structured instruments fit diverse experiences into predetermined categories. That is what makes the results comparable, and it is also what they lose. The data-type consequences are covered in Attribute, Ordinal and Numerical Data.
Selecting a method using the source's own questions
The source gives no procedure for choosing between the thirteen methods. It does, however, ask a set of questions about data at the point where you settle your research question, and those questions function as a selection screen if you use them that way.
- What sort of information do I need — anecdotal, historical, multiple sources?
- Will I use primary and/or secondary sources of data?
- How much data or information is enough?
- What authority do I need to collect this data or information?
- How current does this information need to be?
- What do I need to find out to answer the questions, and where can I find it?
Two further source statements complete the screen. Methods are only as good as the answers they give to research questions, so the methods you choose depend on the questions you want to answer. And your review of the literature gives you the material from which you develop the questions you will ask in data gathering — so the literature work is an input to instrument design, not a separate exercise. See Step 4: Deciding Your Research Question.
A first-pass screen built from the source's questions
Specifying a method so a reviewer can check it
Because the source defines none of the thirteen, the burden of precision falls on your specification. Record the following for every collection method you intend to use. Completing this table is also how you discover that a method you assumed was straightforward is not yet designed.
THE SPECIFICATION RECORD FOR EACH COLLECTION METHOD
| What to record | Why it is needed |
|---|---|
| The method name, with a cited definition | The source defines none of the thirteen; your definition and its citation carry the weight |
| Primary or secondary | The source's distinction, and the basis of how much you must say about provenance |
| The research question it serves | Methods are only as good as the answers they give to research questions |
| What the instrument actually is | A questionnaire is not a method until the questions exist; drafting exposes what you have not decided |
| Who it collects from, and how they are selected | Selection is where quantitative claims about generalising are won or lost |
| How much is enough, and why | The source asks this question directly and it is rarely answered in advance |
| The authority required, and whether you have it | One of the source's own data questions, and a common cause of stalled designs |
| The ethics screen for this method | Confidential or personal information, cultural appropriateness, and unbiased collection |
| How the resulting data will be analysed | The supplied source defers analysis technique to a text it does not reproduce; name yours |
The record is a synthesis. Its individual items are drawn from the source's own questions about data, its ethics screening questions and its statements about methods; the source does not present them as a specification format.
The ethics line is not a formality for research inside your own organisation. The source asks whether the project requires access to confidential or personal information, whether the questions are appropriate and respectful of cultural norms, and whether the collection methods are unbiased and ethical — then defers the substantive treatment; see Step 6: Seeking Ethics Clearance.
What the exemplar designs show about method choice
The published volume of master's project management research supplies something the teaching source does not: seventeen worked designs showing which collection methods practitioners actually used.
Reading across those designs, survey and interview dominate, and almost every design pairs a literature review with one or two primary collection methods. This is an observation about the supplied volume, not a published statistic, and it is not a recommendation — it is a description of what master's-level project management researchers found workable within their constraints. The designs are set out in Seventeen Worked Research Designs in Project Management.
The practical reading is that a modest number of well-specified methods beats an ambitious collection plan. If experienced practitioners with organisational access settled on a survey plus interviews, a part-time researcher planning four methods should ask which two are carrying the argument.
Before you collect anything
The pre-collection check
- Every method you plan to use has a definition you can cite from a source you have read
- You can state, for each method, what case it helps establish or what phenomenon it explores
- The instrument exists in draft, not just in description
- Selection of participants, cases or documents is written down as it will actually happen
- You have answered how much data is enough, in numbers or in a stopping rule
- The authority to collect has been confirmed, not assumed
- The ethics screening questions have been answered for each method
- The analysis approach is named, with the text you are following
Collection sits inside the wider design decision, which is covered in Step 5: Designing Your Research.
One more row moves
The status table above recorded sampling as named and not taught. A later week supplies six sampling types with usable distinctions for three of them, which moves that row — though only as far as vocabulary. No procedure for executing any of the six is given, and no sampling frame appears anywhere in the supplied material.
Three of the thirteen names now have a worked instance behind them
This page's complaint has been that the supplied teaching source names methods and attaches nothing to them. Eight examined research works have since been read, and three of the thirteen names are now demonstrated end to end by somebody who actually ran them — including the one this library had recorded as named in a learning objective and taught nowhere across fourteen weeks.
FROM A NAME TO A WORKED INSTANCE
| Method | What the teaching source gave it | What an examined work now supplies |
|---|---|---|
| Focus group | A name in a list, and a mention in one learning objective | Recruitment through a standing governance body, a facilitation tool with conditional routing, a size decision made at the floor of a cited range, a first sitting abandoned at two attendees, and thematic analysis against four pre-set themes |
| Systematic literature review | A name, and a warning that it is not the same as a literature review | Twelve printed search strings, a twenty-stage process, counts at every stage that produced records, and a flow that reconciles — plus four of its own six exclusion criteria broken |
| Delphi technique | A name | A two-round hybrid of interview and survey, a stated target sample achieved, the complete raw dataset printed, and a counter-intuitive finding reported against the researcher's own expectation |
| Questionnaire / survey | One paragraph of principle | An instrument transcribed question by question with every response option, its consent header, and ten demonstrable design defects |
Four rows, four examined works. Each instance is one project's practice, not a standard for the method.
What to carry forward
- Three of the thirteen named methods now have a worked instance from an examined work — focus group, systematic review and Delphi. Observation and the pilot study remain names with nothing attached.
- The source's one principle is that data constitutes evidence — use it as a test, not a slogan.
- Thirteen collection methods are named on the pyramid's apex and none is defined; this is the largest gap in the supplied material.
- The source does define methods, primary data, secondary data and how quantitative collection works. Use what is there.
- Its questions about information type, quantity, authority and currency form a workable selection screen.
- Specify each method in writing — definition, instrument, selection, quantity, authority, ethics and analysis — because the source will not do it for you.
- Instrument design, sampling, analysis technique and ethics all have to be sourced elsewhere and cited.
Frequently asked questions
What does the supplied source say about data collection methods?
One paragraph: in the context of a research project data constitutes evidence, every researcher is exploring a phenomenon or attempting to establish a case, and to do that the researcher gathers data. Beyond that paragraph the material consists of thirteen method labels on the methodological pyramid and the definitions of primary and secondary data. There is no definition of any of the thirteen methods.
Why does this page not define interviews, questionnaires or focus groups?
Because the supplied source does not, and this library does not invent definitions for terms its source only lists. Definitions of those methods are available in the cited research methods texts, and the right move is to take one from a text you have read and cite it in your methodology chapter.
Is a survey a data collection method or a methodology?
On the pyramid, questionnaires sit on the top tier as a data collection method while survey sits on the middle tier as a methodology. The supplied source does not explain the relationship or define either term. Keep the two separate in your writing: the methodology is the strategy of the study, the method is how the data is gathered.
How much data is enough?
The source poses this question but does not answer it, and gives no sample size guidance. Answer it yourself at design time, either as a number with a justification or as an explicit stopping rule, and record the reasoning. An unanswered quantity question usually means the research question is not yet tight enough to design against.
Can I rely on data my organisation already holds?
Often, but the source's definition of secondary data sets the terms: it was gathered by other people or organisations for other purposes, and it has been interpreted by others. Record what it was originally collected for, who interpreted it and when, and treat those as limitations rather than footnotes.
References and source attribution
- Quinlan, C. 2011, Business Research Methods, 1st ed., Cengage Publishing, Chapter 1, p. 50 — the source of the methodological pyramid, whose apex tier carries the thirteen data collection labels.
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford.
- O'Leary, Z. 2017, The Essential Guide to Doing Your Research Project, 3rd ed., Sage Publications, London.
- Naoum, S. G. 2013, Dissertation Research & Writing for Construction Students, 3rd ed., Routledge.
- An edited volume of master's degree research projects in project management, 2018, academic publisher, 378 pp. — the exemplar volume whose seventeen designs are referred to on this page.
- The supplied teaching source: consolidated week 2 and week 3 teaching notes and slide material on research methods, used here as the basis for all statements attributed to "the source".
Suggested questions for Ask KEVOS
- Build me a specification record for the data collection methods in my research design.
- Which of the thirteen methods on the pyramid apex suit a question about how project teams make decisions?
- Help me answer "how much data is enough" for an interview-based study.
- What is the difference between a data collection method and a methodology on the pyramid?
- Draft the ethics screening answers for a questionnaire issued inside my own organisation.