Research Design and Data Quick Reference
Four weeks of material reduced to what you can act on: the definitions the source gives, the lists it sets out, the numbers it states — and, marked as such, the terms it uses and never defines. Keep it open while you draft your methodology.
How to read every entry on this page
A working index, not an argument. Each definition, list, rule and number is reproduced as the material states it, then classed. The class matters more than the content: a 15 per cent response rate offered once as a rule of thumb and a 600-word activity length stated as an instruction are different kinds of claim, and treating them alike is how a research design acquires a standard nobody set.
THE FOUR CLAIM CLASSES USED THROUGHOUT
| Class | What it means here | Instance |
|---|---|---|
| Source requirement | Stated as something you must do, include or not exceed. | “a copy of each data collection method used in the appendices” |
| Source example | Explicitly an illustration — “for example”, “e.g.” — never a rule. | A study measuring student turnover, given in place of a definition of validity |
| Illustrative value | A number, count, page reference or threshold. A few are stated; most are counts of a printed list the source never states. | Ten methods in a grid the source neither numbers nor counts |
| General principle | Asserted as generally true, with no procedure, criterion or figure attached. | “All of the elements of the research project have to fit together” |
A fifth entry recurs below: “Not defined in source”. It means the material names the thing and defines it nowhere across the four weeks.
Research design: what the first week defines, and what it only names
DESIGN VOCABULARY — DEFINED, AND NOT
| Term | What the material supplies |
|---|---|
| Validity | The slide opens “Validity means, for example…” and supplies only an instance. |
| Reliability | “the extent to which a research instrument, such as a survey, will produce the same results if the research was repeated”; said to have more application to quantitative than qualitative research. |
| Rigour | “establishing the soundness and the dependability of their research” — defined by two further terms, of which soundness is defined nowhere. |
| Trustworthiness | Named once, beside validity and reliability, in a selection question. |
| Generalisability | Named once; the question assumes you know what it is. |
| Focus | Raised to one of two “key issues” in choosing a methodology, never distinguished from “aim”, never seen again. |
| Mixed methods | The slides titled “Mixed Methods Approach” define triangulation instead — which the same slide calls one of the first rationales for mixed methods, so expressly not the same thing. |
| Triangulation | Two definitions in consecutive sentences: “the use of more than one approach to answering the research question”, and “looking at the problem… from more than one perspective”. Three kinds named — researcher, theoretical, method — none defined. |
The methodology–methods relationship takes three incompatible structures across the four weeks: methods inside methodology on one slide, parallel to it on two others, and one of five components of a design in the qualitative week's notes.
Treated in full on what research design is, methodology decisions, methods and their place in a design, validity and reliability, the context of data gathering and mixed methods and triangulation.
The nine method-selection questions (five on one slide, four on the next)
- Does the hypothesis point you in a certain direction for obtaining sources of data?
- Is there any previous research or literature that would indicate an appropriate research method?
- Can you access specific data archives, specific data or people? If not, does this preclude some methods?
- Resources — do you have money to survey people, access to people?
- Time — will this research be done within the schedule of your study?
- Validity, reliability, trustworthiness — will the research be considered valid, reliable or trustworthy when reviewed?
- Generalisability — the ways in which you hope to generalise results from one setting to a larger number of settings.
- Ethics — are there any ethical issues attached to the choice of research method?
- Uses and users of the findings — will this research be useful and add to knowledge?
Class: source examples of the considerations, not a mandatory checklist. The lead-in is “ask yourself these questions” and no count is stated. The same slide names five dependencies — type of topic, skills and experience, availability of access, time factors, ethical matters — which overlap the nine without matching them, and neither list is said to supersede the other. Both on selecting a research method.
THE TEN-METHOD GRID, REPRODUCED AS LAID OUT
| Column 1 | Column 2 | Column 3 |
|---|---|---|
| Structured interviewing | Content analysis | Semi- or unstructured interviewing |
| Self-completion questionnaires | Secondary data analysis | Focus groups |
| Structured observation | Ethnography / participant observation | Conversation or discourse analysis |
| Documentary analysis | — | — |
Ten populated cells, two empty; no count and no numbering in the source. Not one is defined, described or compared on any dimension anywhere — no benefits, no disadvantages, no data types, no use cases. The ten-method catalogue sets out what that costs.
Qualitative research: five statements of one definition, and the sets that follow
HOW THE MATERIAL DEFINES QUALITATIVE RESEARCH — ALL FIVE STATEMENTS
| Where | The definition as given | What it makes qualitative research |
|---|---|---|
| Slides, from a 2012 general dictionary | “Data-gathering techniques that are focussed on the significance of observations made in a study rather than the raw numbers themselves” | A class of procedures |
| Notes, from a 2009 guide, p. 13 | understanding “the meaning people have constructed… how people make sense of their world” | A purpose and focus |
| Notes, from a 2011 source | “research using methods such as participant observation or case studies which result in a narrative, descriptive account” | An epistemological stance |
| Notes, from a 2005 handbook, p. 3 | “a situated activity that locates the observer in the world… an interpretive, naturalistic approach” | A situated practice |
| Notes, own position | “as much a perspective on how to approach investigating a research problem as it is a method” | A perspective |
CONFLICT. The slides define a technique; the notes define a perspective and quote three specialist definitions, none of which defines a technique. Unresolved in the source. The slides also define by negation — non-numeric, “not subjected to rigorous quantitative analysis” — while all three quoted definitions are positive. See what qualitative research is and what qualitative researchers believe.
Two count conflicts sit inside the same week. The slides say the approach “takes a why, how, when, what approach”; the notes say qualitative questions are “‘How’ or ‘What’ questions” and supply worked examples for that pair only. And the slides' three typical techniques — focus group interviews, projective techniques, depth interviews — share nothing with the same deck's own five commonly used methods.
THE ENUMERATED SETS, AND WHETHER THEIR MEMBERS ARE DEFINED
| Set | Members | Count stated? |
|---|---|---|
| Five components of a qualitative design (2009 source) | Research goals; conceptual framework; research questions; methods; validity. None of the five is defined; they are named in one sentence and never returned to. | Yes — five stated, five listed |
| Ten strengths of qualitative methods | From a realistic view of the lived world to a descriptive capability based on primary and unstructured data. Stated as capabilities, not defined. | No — “Among the specific strengths”, explicitly non-exhaustive |
| Two limitations of qualitative enquiry | Small samples undermine generalisation; researcher embeddedness increases bias in how data is gathered, interpreted and reported. No mitigation for either. | No — “Most of the limitations… also reflect their inherent strengths” |
Eight of the ten strengths receive no corresponding limitation. See the five components of a qualitative design and strengths and limitations of qualitative research.
THE FOUR ALTERNATIVE CRITERIA FOR JUDGING QUALITATIVE RESEARCH
| Criterion (against) | Definition as given | Procedure |
|---|---|---|
| Credibility (internal validity) | “believable from the participant's perspective” | None |
| Transferability (external validity) | “the degree to which the results… can be generalised or transferred to other contexts or settings” | Describe context and assumptions thoroughly — but generalising “is the decision of the person wishing to do so” |
| Dependability (reliability) | account for “the changing context… and how these changes affect the way the researcher approaches the study” | None |
| Confirmability (objectivity) | “the degree to which the results can be confirmed or substantiated by others” | Three, all hedged with “can”: document checking and rechecking; search for negative instances; conduct a data audit |
All four are defined — the only place in the week where a named set is fully defined. CONFLICT: the same deck states “There is no point in trying to establish ‘validity’ in any external or objective sense”, then devotes three slides to these criteria; and the notes make validity one of the five components of every qualitative design. See validity criteria for qualitative research and reflexivity and the researcher's position.
TERMS THE QUALITATIVE WEEK DOES DEFINE, IN ITS OWN WORDS
| Term | Definition as given, and what it omits |
|---|---|
| Ethnography | “study of the culture of a defined group”; most common approach is participant observation. Equated with participant observation in the design week's grid, nested inside it here. |
| Phenomenology | “emphasizes individual interpretation of experiences and perceptions of reality”. Eight words; no method, data source or example anywhere. |
| Field research | “a broad approach to qualitative research or a method of gathering data” — two definitions in one sentence, with no basis for choosing. |
| Grounded theory | “theory is grounded in observation – it is a complex iterative process”, with new observations leading to revisions and more data collection. Circular, and the loop has no stopping rule. |
| Participant observation | “the researcher becomes part of the context being observed”. Four issues named — entering the context, the participant role, collecting and storing field notes, analysis — and none answered. |
| Direct observation | the researcher “doesn't become a participant… but tries to remain unobtrusive”. Said to take less time than participant observation; no duration for either. |
| Unstructured interviewing | “direct interaction between the researcher and respondent(s)”, with “no formal structured instrument or protocol, although there are some guiding questions”. Structured interviewing, the basis of the contrast, is defined nowhere. |
| Case study | “an intensive study of a specific context or person”. No selection criterion, number of cases or boundary rule. |
| Documentation | organisational records “including minutes, official publications and reports, journals, letters, artistic works, photographs, and written responses to open-ended surveys”. The restriction and the examples do not agree. |
| Thick description | attention to “rich detail, meaningful social and historical contexts and experiences, and the significance of emotional content”. Quoted from a 1989 source. |
| Memos | notes, “or possibly sketches”, made as collection and analysis proceed, tracing the researcher's thinking. Field notes, named four times, are never defined, and the source never says whether the two are the same. |
| Analytic framework | “a network of linked concepts and classifications” used to understand an underlying process — “a sequence of events or constructs and how they relate”. |
Five methods on five qualitative methods; the analytic vocabulary on thick description, memos and field data.
The ten-item qualitative design checklist, in order
- Determine the extent to which qualitative methods are appropriate given the project's purposes and intended uses.
- Determine which general strategic themes of qualitative inquiry will guide the project.
- Determine qualitative design strategies, data collection options, and analysis approaches based on the project's purpose.
- Determine which qualitative applications are especially appropriate given the project's purpose and priorities.
- Make major design decisions so that the design answers important questions for intended users.
- Where fieldwork is part of the project, determine how to approach the fieldwork.
- Where open-ended interviewing is part of the project, determine how to approach the interviews.
- Design the project with careful attention to ethical issues.
- Anticipate analysis — design the data collection to facilitate analysis.
- Analyse the data so that the qualitative findings are clear, credible, and address the relevant and priority project questions. Focus the report.
Class: stated to be adapted from an evaluation project, with the adaptation asserted rather than described. Its three numbered purposes are never mapped to its ten unnumbered items; it is unattributed; and items 6 and 7 defer the instruction rather than supplying it. Full treatment on the qualitative design checklist.
Gathering your data: the question sets, the rules and the response formats
THE SIX APPROACHES TO COLLECTING QUALITATIVE DATA
| Approach | Definition as given | Methods attached |
|---|---|---|
| Phenomenology | “Focuses on individual experiences, beliefs, and perceptions.” | In-depth interviews and focus groups, called “ideal” |
| Ethnography | “Oriented toward studying shared meanings and practices (i.e., culture).” | Participant observation “well suited”; traditionally long-term fieldwork |
| Inductive thematic analysis | “Involves identifying and coding emergent themes within data.” | In-depth interviews and focus groups; requires “free-flowing data” |
| Grounded theory | “Inductive data collection and analytic methods.” | Exhaustive comparison of text segments; sample sizes “more limited” than for thematic analysis, neither term quantified |
| Case study | “Analysis of one to several cases that are unique with respect to the research topic.” | Cases selected on a unique, often rarely observed quality |
| Discourse analysis | “Study of ‘naturally occurring’ discourse.” | Conversation, public events, existing documents |
CONFLICT. The qualitative week gives four approaches — ethnography, phenomenology, field research, grounded theory — this week gives six, three new and one dropped, and the three shared items are defined differently in the two weeks. Three of these six describe themselves as analytic approaches inside a section introduced as being about collection. See six qualitative data collection approaches.
THE FOUR QUESTION SETS IN THE DATA COLLECTION WEEK
| Set | N | What it covers |
|---|---|---|
| “Ask yourself the following questions” | 8 | Purpose fit; whether the information reaches conclusions; time for analysis; validity and reliability; cost and time; the what/how/when/who of implementation; management and analysis; data security and storage. Ethics is absent, though the design week asks about it. |
| “First of all, ask yourself” | 9 | Content, scope and purpose; response format; population knowledge; reading skills; wording; facilities and timeframes; complexity; sensitivities such as age and income; methods used in similar projects. |
| “Ask yourself these questions” | 6 | Confusing or misleading; assumptions; time-frame; sensitivity; jargon and acronyms; whether the words define what you want to know. |
| “How will the information be collected and analysed?” (notes) | 5 | Data sources; data types; collection methods; how data will be analysed and presented; ethical issues. |
Twenty-three self-directed questions across three slides, with no consolidated checklist, no ordering and no statement of which apply when. All are source examples of considerations, not pass marks. Question 4 of the five is never answered anywhere in the week. See the data collection and analysis sequence, designing a questionnaire and the wording of questions.
SURVEY AND QUESTIONNAIRE RULES, AS STATED
| Rule set | Content and class |
|---|---|
| Five steps in data collection (2011 statistics text) | Define the variables; collect; organise; present in relevant form; analyse. Source sequence — and it puts presentation before analysis, which the quantitative week reverses. |
| Survey research | “includes any procedures that involve asking questions of respondents… from a written evaluation to an in-depth interview”. Four properties: descriptive or explanatory; entire populations or samples; capture a moment or map trends; administered in a number of ways. Broad enough to swallow interviews and focus groups. |
| Survey construction | Four activities: questions and response categories; background information and instruction; organisation and length; layout and design. No length is ever stated. |
| Six design points that “should be observed” | Short and simple; unambiguous; as short as possible; neither irrelevant nor too personal; not leading; in a logical sequence. Two of the six are the same requirement, so the distinct points are five. |
| Five reminders | Sequence appropriately; clear layout; not misleading; familiar language; thank the respondent. Four restate earlier instructions. |
| Postal questionnaires: “not recommended unless one of the following applies” | Completion is a legal obligation, e.g. census; all non-responders are interviewed; an appropriate sample of non-responders is interviewed and failure to reply is established as “in no way connected with any bias”. No size for the “appropriate sample”, and the same deck says returned questionnaires are frequently biased. |
The words “pilot”, “pre-test” and “trial” appear nowhere in the week. Its entire pre-testing guidance is one clause: wording should be “rehearsed with someone else before you begin”. See survey research design and questionnaires and response rates.
LEVELS OF MEASUREMENT AND RESPONSE FORMATS, EXACTLY AS SUPPLIED
| Type | Definition given | Example | Coding shown |
|---|---|---|---|
| Dichotomous | Four instances only | yes/no; agree/disagree; true/false; male/female | None |
| Nominal | None | language spoken at home | 1 = English, 2 = French, 3 = Chinese |
| Ordinal | “ranked in order” | preferred travel destination | None |
| Interval | None | a five-point agreement scale | 1 = strongly agree … 5 = strongly disagree |
| Ratio | Not named anywhere in the four weeks; the slide does not say the list is partial | — | — |
CONFLICT, and it matters. The scale filed under Interval is, on the slide's own definition of Ordinal, ordinal — it is ranked in order. The property that separates them, equal spacing between adjacent points, is never mentioned. The slide defers its definitional content with “(mentioned in topic 7)”, and the words nominal, ordinal, interval and level of measurement appear nowhere in the design week. See levels of measurement.
QUESTION TYPES AND INTERVIEW TYPES
| Item | As stated, and the conflict to carry |
|---|---|
| Structured / unstructured questions | “Structured: respondent writes reply”; “Unstructured: interviewer writes reply”. This is the questionnaire-versus-interview distinction renamed. Under it every questionnaire is structured and no interview can be — which the same deck denies twice, naming “Structured open-ended interviews” and titling a slide “Choices of responses in structured questionnaires”. |
| The advantage table above those lines | Quick and easy to reply and efficient to summarise, against constrained responses and no deep understanding; rich deep information and a range of leads, against time, cost and difficulty in summarising. Neither block is labelled: the attribution must be inferred, and the inference flagged rather than stated as given. |
| Interviews range from | Formal to informal; structured to unstructured; one-on-one or groups. The four named types are not positioned on any of the three ranges. |
| The four types | “Structured open-ended interviews provide standard questions to give each respondent the same stimulus”; “Interview guide approaches identify the topic but not wording of questions”; “Conversational interviews are interactive”; “Focus groups occur when the interviewer becomes a facilitator in a group of respondents”. |
| What those four omit | “Interview guide” appears once in the whole supplied source and no guide is shown or specified. Focus groups get one sentence, for a technique named in the week's only objective and recommended nine times in its notes — no group size, recruitment, moderation, recording or analysis. |
Interviewing is defined four different ways across four slides of one deck and no slide acknowledges the others. See structured and unstructured questions and interview types and selection.
CONDUCTING THE INTERVIEW — THE FOUR INSTRUCTION SETS
| Slide | N | Representative content |
|---|---|---|
| Considerations for interviewers | 5 | Establish credibility and trust; listen carefully; practise; observe the respondent carefully; use probes without getting side-tracked. “Probes” is not defined in the source. |
| Conducting interviews | 10 | Adapt technique to the person; explain purpose and risks; keep promises; ensure confidentiality; ensure anonymity; obtain informed consent; explain who has access to the data; protect the data |
| Preparing for the interview | 8 | Confirm venue and time; be punctual; notebook and pen; copies of the survey instrument; written project information; official identifier; a cover letter from your supervisor |
| At the interview | 11 | Professional appearance; rapport; short explanation; easy questions first; memorise the first few questions; stay with the script if using structured questions; silence as a probe; ask for elaboration; offer no opinions |
The ten-instruction slide is the densest concentration of ethics obligations in the four weeks — confidentiality, anonymity, informed consent, risk disclosure, data access, data protection — and none of the six is defined, no consent form or elements are shown, and no approval process is mentioned. Recording guidance is three lines and a consent caution; note-taking guidance is “Develop your own short-hand system”; no transcription procedure or notation appears anywhere. See conducting and recording interviews.
Quantitative analysis: the vocabulary, the one formula and the measure that never arrives
THE FIVE KEY TERMS AND THE ANALYSIS VOCABULARY, AS DEFINED
| Term | Definition as given, and the note to carry |
|---|---|
| Data management | “becoming familiar with appropriate software, logging in, entering data and cleaning data”. CONFLICT: the data collection week defines the same phrase as “The process of data collection and analysis”. The two do not overlap; neither document acknowledges the other. |
| Data screening / cleaning | “identifying any gaps, errors, incomplete questions, misclassification or illegible responses”. Defined again on the next slide as identifying “any errors”, with four different categories — internal consistence, miscoded, missing, messy data. Nine defect labels, no canonical set. |
| Coding keys / code book | “Guide to all of the codes used in coding data to input data into a computerised software program”. Defined once and never used again, including on the two slides devoted to coding. |
| Data coding | “the transformation of data into a form understandable by computer software”. The same slides also describe refining data into categories and sub-categories, an interpretive operation. Category, sub-category and code are undefined and unexemplified. |
| Outlier | “one that appears to deviate markedly or is very distant from other members of the sample”. Purely distributional; says nothing about error. Three later slides characterise it three further ways. No detection rule of any kind is given. |
| Descriptive statistics | “Describes distribution of relationship among variables”. Grammatically broken and open to two readings; do not silently choose one. |
| Inferential statistics | “Used to estimate how likely it is that a statistical result based on data from a random sample is representative”. No inferential procedure is ever named — no test, no interval, no estimation method. |
| Central tendency | “Most common value for variables measured at a nominal level”. CONFLICT: that is the mode, in a table that separately defines the mode. Central tendency is the general concept of which mean, median and mode are the measures. |
| Base number | “Total number of cases in the distribution”. Defined and never used again. |
| Frequency distribution | “Numerical display of the number of cases corresponding to each value or group of values of a variable”. |
| Mean | “The weighted average calculated by totalling the value of all the cases and dividing by the total number”. CONFLICT: the label names a weighted average, the procedure is the arithmetic mean. No weight is mentioned anywhere. |
| Median | “The point that divides the distribution in half”. No procedure: no instruction to order the values, nothing on even-numbered samples, nothing on whether it must be an observed value. |
| Mode | “Most popular value”. Cannot express a distribution with two peaks, which a later slide asks you to look for. |
| Statistically significant | “unlikely to have happened by chance”. A nine-word gloss filed as one of sixteen “issues in data analysis”. No significance level, p-value, alpha, null hypothesis or test is named anywhere in the week. |
Fifteen of those sixteen issues are bare labels with no elaboration anywhere. See coding quantitative data, data screening and cleaning, handling outliers, measures of central tendency and descriptive and inferential statistics.
PRESENTATION RULES, THE ONE FORMULA AND THE ONE THRESHOLD
| Rule | As stated | Class |
|---|---|---|
| Effective digits | “The basic requirement is to keep the number of effective digits… to two or three”, restated a line later as “(often 2-3)”. | CONFLICT: a requirement and a frequency, two lines apart. Reproduce both. |
| Seven table rules | Appropriate effective digits; averages for rows and columns; order rows and columns by size; put the main comparison in a column; avoid intricate grids; use labels; give a brief summary. | Source requirements. Which of the three averages to report is never stated. |
| Table typology | Univariable, bivariable, multivariable, by number of variables. | Source classification. “Cross-tabulation” appears nowhere. |
| Percentage treatments | Straight percentages, accumulated percentages (Pareto), ratios. | Neither Pareto nor ratios; no percentage figure appears anywhere. |
| Graph principles | Label axes and title the graph; make the data the feature; “the more ink you use… the less effective the representation is”. | Source principles. The data-to-ink ratio is given a direction and never defined as a ratio. |
| Histogram class count | classes “equal to approximately the square root of the number of observations”; “25 observations… suggests approximately 5 classes”. | The only formula in the week, with its only worked example. A rule of thumb, not a requirement. |
| Histogram threshold | “Histograms are not so effective for less than about 20 observations.” | Double-hedged. The only numeric sample-size guidance in the four weeks — and it governs observations on a chart, not participants in a study. |
| The empirical rule | 68% within one standard deviation of the mean, 95% within two, 99.7% within three. | Quoted from a 2005 text, p. 462. Stated as absolutes about “all observations”, without the normality condition that makes them true and without the word “approximately”. |
| Sketching a normal distribution | Draw the bell curve; mark the centre with the mean; mark off three standard deviations either side, the standard deviation “here” being 5 units. | Source requirement for the drawing. The 5 units belongs to no stated dataset, variable or mean, so the axis cannot be drawn from the slide's own information. |
Both illustrated figures are unavailable in the supplied files: the histogram slide carries the literal placeholder “Images for histogram”, and the bell-curve slide is a single image from which no text or value is recoverable. See presenting data in tables, presenting data in graphs and the normal model and the empirical rule.
READING A DISPLAY, AND THE THREE FEATURES OF A DISTRIBUTION
| Set | Members | Count matches its list? |
|---|---|---|
| Four key aspects when looking at graphical representations | Overall shape — symmetrical, or a hump on one side and a longer tail on the other, “skewed in the direction of the tail”; one clear peak or more, “may be” a mixture of data from two or more sources; points obviously different from the main pattern, called outliers, which “often point to some set of special circumstances which could be quite valuable to understand”; clusters, again possibly different sources. | Yes. One of only two places in these two weeks where a stated count and its list agree. |
| Three key features of a distribution | Shape, location and spread. | Yes. The second of the two. |
| Definitions on the same slide | Distribution: “a collection of measurements taken on an aspect of interest within a particular situation”. Parameter: “a feature of the model such as location or spread”. Bell-shaped curve: the symmetrical smooth curve drawn over a distribution, “the basis for the normal probability model”. | No counts. “Statistic” is never introduced, so the parameter/statistic distinction is unavailable. |
| Variation and variability | “the lower the variability the higher the consistency”, illustrated by noticing subtle differences in the same restaurant dish ordered on several occasions. | No count. The example carries no numbers at all, and the two words are used interchangeably without either being defined operationally. |
“Location” here is the same concept the vocabulary table calls “central tendency”, and the deck never says so — with two incompatible glosses, one pointing at the mean and median, the other at the mode. “Bimodal” and “multimodal” are never used. See reading a distribution.
Where the material says one thing two ways
These are the conflicts most likely to trip you while drafting. Both statements are reproduced, neither preferred, and the source reconciles none of them. If a marker or supervisor asks which you followed, the useful answer names both and says which you used and why.
THE CONFLICTS THAT AFFECT WHAT YOU WRITE
| Question | One statement | The other |
|---|---|---|
| What is qualitative research? | A class of data-gathering techniques (slides). | A perspective on how to approach a research problem, as much as a method (notes). |
| What is data management? | “The process of data collection and analysis”. | Software familiarity, logging in, entering data and cleaning data. |
| Where do methods sit? | Inside methodology — “a major part of the methodological framework”. | Parallel to methodology; and separately, one of five components of a design. |
| Does validity apply to qualitative work? | “There is no point in trying to establish ‘validity’ in any external or objective sense.” | Three slides of alternative criteria, plus notes making validity one of the five design components. |
| What is dependability? | Part of rigour, offered as the substitute for consistency of results. | Accounting for the changing context and its effect on the researcher's approach. |
| What makes a question structured? | Who writes the reply. | Whether response options are fixed — the sense the same slide's own advantage table uses. |
| What is central tendency? | “Most common value for variables measured at a nominal level” — the mode. | “Location”, glossed seven slides later as “perhaps an average or median”. |
| What do you do with an outlier? | “Such an obviously erroneous figure will be removed”; wrongly including an extreme value is “a more serious error than wrongly ignoring one”. | Outliers “often point to… special circumstances which could be quite valuable to understand”; and the decision “is left to the researcher's judgement”. |
| Present then analyse, or analyse then present? | Five steps: organise, present, then analyse. | The quantitative week: analysing data, then presentation. |
| How fast should you choose a methodology? | “decide as quickly as possible”. | “take time to get this right” — and, a slide earlier, for the beginner “expect the decision-making process to be slower”. |
| Is qualitative design planned or emergent? | A ten-item checklist requiring strategies, options and analysis approaches to be determined in advance. | Design “emerges”; measures are “created in an ad hoc manner”; questions “will emerge and change”; collection and analysis “usually proceed simultaneously”. |
| How free are you to choose a method? | “How you conduct your study is entirely up to you” (notes). | Eight gating questions, a prohibition on postal questionnaires, and six design points that “should be observed” (slides of the same week). |
| Which quantitative/qualitative comparison table governs? | Four rows from a 2010 guide, p. 105; seven rows from a 2006 text, p. 157; five rows, unattributed. | Different dimensions, row counts and vocabulary. No document mentions the others; none is stated to be authoritative. |
Before you quote: one citation's page range runs backwards, an unfilled editing placeholder survives in a worked example, three diagrams are supplied as images with no recoverable content, and the prescribed text is referred to throughout as “Text”, its author surname appearing once, incidentally, on the last content slide of the last week.
Every number across the four weeks, by class
THE NUMBERS, CLASSED
| Value | What it is | Class |
|---|---|---|
| 225 / 300 / 375 / 600 / 400 words | Activity and assignment-component lengths across the four weeks | Requirements. 1,500 words as the assignment total is derived by addition, not stated. |
| Ch. 3 pp. 44–77 · Ch. 26 pp. 611–625 · Ch. 24 pp. 564–588 · Ch. 15 pp. 329–351 · Ch. 16 pp. 353–374 | Set readings from a prescribed text that is never identified | Requirements |
| “all of Chapter 8” | A further design week reading | Requirement whose book is not stated and belongs to neither reading list |
| A copy of each data collection method in the appendices | Proposal and report obligation | Requirement, without exception and without a page limit |
| Completable within the study period | Constraint on the chosen methodology | Requirement; the study period's length is never stated |
| Two or three effective digits · three standard deviations either side | Table rounding; the normal-curve sketch | Requirements |
| 15% | “A return of 15% on postal surveys may be a good response” | Unattributed rule of thumb, hedged twice. Never a benchmark. |
| 34 years · 62 metres | An age among year-one children; a height among men | Examples of self-evident implausibility, not of any detection rule |
| $10,000 · $1,000 | A survey average of money brought onto a cruise ship, and the maximum after removing one respondent | Examples. No sample size, no arithmetic, so the case cannot be checked. |
| 25 observations → about 5 classes · about 20 observations | The histogram class rule and its threshold | Example and threshold, both hedged |
| 5 units | “the standard deviation being used here is 5 units” | Example whose dataset, variable and mean are never stated |
| 68% · 95% · 99.7% | The empirical rule | Quoted from a 2005 text, p. 462 |
| 1 = English, 2 = French, 3 = Chinese · 1 to 5, strongly agree to strongly disagree | Nominal coding; a response scale labelled “Interval” | Examples. The second is filed under the wrong level on the slide's own definitions. |
| Age and income · “XYZ soap” · “the next year” · census · student ID and business card | Sensitive topics; a leading question; a question time-frame; a legal-obligation questionnaire; official identifiers | Examples |
What a second source has to supply before you can proceed
The shortest useful summary of these four weeks: they will get you to a defensible statement of what you intend to do, and they will not get you to a working instrument or a completed analysis. Below is what to find, grouped by the point at which you will discover you need it.
THE SECOND-SOURCE LIST, BY WHEN YOU WILL NEED IT
| When | Not supplied | Where to go |
|---|---|---|
| Choosing a methodology | Descriptions of the seven methodologies named and deferred; any criteria or example of the justification required. | Covered from the earlier weeks of the same source: the methodology catalogue and choosing a research methodology. |
| Choosing methods | Any benefit, disadvantage, data type or use case for any of the ten methods. No two methods are compared on any criterion anywhere. | A methods text. Data collection methods in research records what the library holds. |
| Designing your sample | Sampling methods, frames, size rules, non-response handling. Named in an objective and not taught. | A sampling chapter. Nothing here can be inferred from the supplied material. |
| Building the instrument | No questionnaire, interview schedule, observation protocol, focus-group guide or coding sheet is displayed anywhere. No piloting procedure; no transcription convention. | Designing a data collection instrument walks the attempt and shows where the material runs out. |
| Running qualitative fieldwork | How to run a focus group; how to observe; what a field note looks like; what coding is; how to reduce or display data beyond three-word glosses. | A qualitative methods text. The glosses are quoted and carry no procedure. |
| Ethics | Six obligations named on one slide and none defined. No code, principle or approval process in these four weeks. | The earlier weeks do cover it: research ethics quick reference, and covert research and negotiating access for the access case these weeks raise. |
| Analysing quantitative data | Any measure of dispersion; any definition of standard deviation; any inferential procedure; any statistical assumption by name; any missing-data technique; any coding option by name. | A statistics text. These are gaps in the files supplied, not in the subject: that week's study notes were not among them. |
| Presenting results | A worked table; a surviving chart image; a decision rule for choosing among displays; any link between level of measurement and display type; bar, pie, box or Pareto charts. | A presentation or statistics text. Five display types are named with a purpose each, and that is where the material stops. |
What to carry forward
- Read the class before the content. A requirement, an example, a derived count and a quoted page number are four different things, and only one of them binds you.
- “Not defined in source” is the most common entry across these four weeks. Name the gap in your own writing and cite a second source; do not supply a plausible definition.
- Where the material states something two ways — data management, structured questions, central tendency, what to do with an outlier — reproduce both and say which you followed.
- The 15 per cent postal return is unattributable and double-hedged, and it is not a benchmark for any survey you run.
- No participant sample size appears anywhere in these four weeks. The one sample-size figure is a charting rule about observations on a histogram.
- The quantitative week is slides alone because its study notes were not supplied. Attribute its gaps to the material supplied, and go and find the definitions.
Frequently asked questions
Which numbers on this page can I treat as standards?
Only those classed as requirements, and only inside the assessment context that set them — word counts, page ranges, the appendix obligation, the rounding rule and the three-standard-deviation sketching instruction. Everything classed as an example is an illustration the source offered once. The 15 per cent postal return in particular is an unattributed, double-hedged rule of thumb and is not a benchmark for any survey you run.
The material gives two definitions of data management. Which should I use?
State which you are using and record that the source supplies two. The data collection week defines it as the whole process of collecting and analysing data; the quantitative week defines it as becoming familiar with software, logging in, entering data and cleaning data. The two do not overlap, neither document acknowledges the other, and nothing in the material makes one authoritative.
Why does the levels-of-measurement table say the interval example is in the wrong place?
Because on the slide's own definitions it is. Ordinal is defined as “ranked in order”, and the five-point agreement scale filed under Interval is ranked in order. The property that separates the two levels — equal spacing between adjacent points — is never mentioned. This page records the discrepancy and does not correct it, because correcting it would mean attributing a definition to material that does not contain one.
Where is the standard deviation defined?
Nowhere in the supplied material. It is used four times on one slide to express the only quantitative rule in the week, and it is not among that week's five key terms or its analysis vocabulary. That week supplied no study notes, so the absence belongs to the files supplied rather than to the subject as taught. Take the definition from a statistics text and say that is where you took it from.
How many sampling methods does this material teach?
None. Sampling is named in a learning objective and receives one sentence recommending random selection and immediately withdrawing it as usually impossible, with no alternative offered. No sampling frame, method or size rule appears anywhere across the four weeks, and no participant sample size is stated in them either.
Can I build my questionnaire from these weeks?
Partly. You will get construction activities, nine design questions, six design points, five reminders, six wording checks and three response formats, which is enough to critique a draft. You will not get a questionnaire, because no instrument of any kind is displayed anywhere in the four weeks, and you will not get a piloting procedure, because the words pilot, pre-test and trial do not appear.
References and source attribution
- The supplied teaching source: the research design week (study notes and a 20-slide deck), the qualitative research week (two content-identical study-note files and a 24-slide deck), the data collection week (two content-identical study-note files and a 27-slide deck), and the quantitative analysis and presentation week (a 25-slide deck; no study notes were supplied). Every definition, list, rule and number on this page comes from those files.
- Veal, A. J. 2005, Business Research Methods: A Managerial Approach, 2nd ed., Longman. Cited at p. 462 for the 68/95/99.7 rule.
- O'Leary, Z. 2010, The essential guide to doing your research project, Sage Publications. Cited at p. 105 for the four-row quantitative/qualitative comparison table; the 2017 third edition is the edition cited elsewhere in the supplied material.
- Miles, M. & Huberman, M. 1994, Qualitative data analysis: An expanded sourcebook, 2nd ed., Sage. Cited at p. 4 for the two quotations set as the design week's activity, at p. 72 for the fieldwork passage, and for the three analysis activities and their glosses.
- Merriam, S. 2009, Qualitative research: A guide to design and implementation, Jossey-Bass; Denzin, N. & Lincoln, Y. (eds) 2005, Handbook of qualitative research, 3rd ed., Sage; Patton, M. 2002, Qualitative research and evaluation methods, 3rd ed., Sage. The three specialist works quoted in the qualitative week's study notes for its definitions, its data sources and its creativity suggestions.
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford. Standing reference for terms these weeks name without defining.
Suggested questions for Ask KEVOS
- List every term on this page marked “Not defined in source” that I would need before writing a quantitative methodology chapter.
- Draft a methodology paragraph using the nine method-selection questions, marking each claim as a requirement, an example or a general principle.
- Show me both sides of the structured/unstructured conflict and help me word a sentence that cites both.
- Which numbers on this page could I legitimately put in a research proposal, and which would I have to attribute elsewhere?
- Build a pre-flight checklist for my data collection design from the four question sets on this page.
- What exactly does the supplied material say about sample size, and what would I have to source elsewhere?