Five Journal Article Research Designs
Five published papers supplied alongside the teaching material, read for how they were built rather than for what they found. Between them they show a research literature that mostly never asks anyone anything — and a journal house style that shapes a paper before its author writes a word.
Five papers, read for design
THE FIVE DESIGNS AT A GLANCE
| Question it asks | Year | Extent | Design | Where the evidence comes from | |
|---|---|---|---|---|---|
| P1 | Do government R&D subsidies stimulate or displace privately financed R&D? | 2002 | 22 pp, economics journal | Econometric policy evaluation | Existing firm-level data on manufacturing firms in one national economy |
| P2 | How should an R&D project be valued when the firm hedges? | 2008 | 7 pp, practitioner-facing technology management journal | Mathematical modelling | None — the contribution is analytical |
| P3 | What distinguishes high-effectiveness R&D organisations from low? | 2000 | 7 pp, same practitioner journal | Comparative survey | R&D directors, surveyed |
| P4 | How can uncertainty be managed across an R&D portfolio? | 2011 | 10 pp, same practitioner journal | Case study proposing a method | One large R&D investment at one organisation |
| P5 | Are patents efficient when research lines are imperfectly correlated? | 2011 | 16 pp, economics journal | Formal theoretical modelling | None — deduction from stated assumptions |
Design labels are this page's classification of what each paper does. Years and extents are as recorded in the supplied material.
Read the last column first. Three of the five never generate a new observation of the world, and a fourth works from records collected by someone else for another purpose — a very different distribution from the master's work at Seventeen Worked Research Designs in Project Management, where survey and interview dominate.
P1 — building a design around a problem you cannot observe
This is the most instructive sentence across the five papers. P1 does not open with its topic, its importance or its literature. It opens by naming what it cannot observe, and everything after is an argument about how close it can get.
It is equally direct about its gap: innovation policies are perceived as crucial to a sector's success, "yet, there is no quantitative assessment of the effectiveness of these policies. This paper attempts to close the gap."
The reporting is worth copying. The paper distinguishes the small-firm effect from the average rather than letting one stand for the other, and reports a statistically insignificant result as a result. Headlining the figure of 11 and omitting the average is exactly how a defensible study becomes a misleading one.
P2 and P5 — arguing without data
Two of the five collect nothing. They are not weak empirical papers; they are not empirical papers. The contribution is the argument, judged on whether the reasoning holds rather than on whether the sample was adequate.
P2 — mathematical modelling
- Starts from a stated defect in existing practice: conventional real-option methods "may over- or under-state a project's value because they are apt to be influenced by the R&D firm's subjective expectations of the future market or technological prospect"
- Proposes a method incorporating firms' hedging behaviour that "would not be influenced by the arbitrary judgment of project evaluators"
- Structure: an OVERVIEW abstract box, a KEY CONCEPTS line — real options, investment diversification, hedging effect — then a modelling section, then references
- No data collection, no sample, no fieldwork, and no apology for their absence
- The claim is comparative: the proposed method removes a specific dependence that the existing method has
P5 — formal theoretical modelling
- Builds a model of an industry regulated by an authority that can subsidise R&D expenditure
- Two assumptions carry the whole contribution: potential innovators' research lines are imperfectly correlated, and imitation takes time
- Both are relaxations of standard assumptions — that research lines are independent, and that imitation is instantaneous
- Compares market equilibrium with patent protection against equilibrium without patents, and reasons about social welfare
- No data, no sample, no case. The argument is deductive from the stated assumptions
P5 is the cleaner teaching case, because its contribution is locatable. Change two assumptions and the welfare conclusion changes. A reader wanting to challenge it knows exactly where to aim — which is what a well-built deductive argument looks like.
P3 — the survey, and the variable that defines its groups
P3 is the only one of the five that collects primary data from people. R&D directors in high- and low-effectiveness organisations were surveyed and the two groups compared.
Its stated finding is a genuine two-part result: directors in both groups attach similar importance to the skills and knowledge bases needed, but high-effectiveness organisations "are significantly more capable than their low-R&D effective counterparts in almost all areas". Knowing what matters is not the differentiator; being able to do it is.
WHAT WOULD HAVE SEPARATED THE TWO MEASURES
| Design move | What it would have bought | What it would have cost |
|---|---|---|
| Group by an external outcome — patents, launches, revenue from new products | The grouping no longer depends on the respondent's opinion of their own organisation | Access to performance data, and an argument about which outcome measure is fair across organisations |
| Take capability ratings from someone other than the director | The two measures come from different people, breaking the common-method link | A second population to recruit, and a new question about whose rating counts |
| Separate the two measurements in time | Weakens, though does not remove, the tendency to answer consistently within one sitting | Two rounds of contact, and attrition between them |
| Keep the design and state the limitation | Costs nothing, and tells the reader exactly how far the finding travels | Nothing except the willingness to write it down |
Synthesis. The supplied material records the design feature; it does not propose alternatives. The fourth row is what a critique should ask for at minimum.
The transferable point is narrow: where the variable sorting your cases into groups is a self-assessment, you are comparing self-assessments, not the things assessed. That belongs in the design section, not the discussion — see Survey Design and Response Rates in Practice.
P4 — a case study whose output is a method
P4 proposes a project portfolio option-value method and applies it to one large R&D investment at one organisation. The case is not the finding — it is the demonstration that the method runs.
It is unusually clear about its own scope. The method, it states, "is not about 'perfect' or 'complete' valuation models, but rather about providing a comprehensive but not-too-detailed view of major challenges and key criteria for success" — declining a claim in advance, and foreclosing a line of criticism with it.
The four conditions P4 names its method for
Many technological and market uncertainties
The method is aimed at portfolios where the unknowns are numerous rather than at single projects with one dominant risk.
Resolution order cannot be specified in advance
You cannot say up front which uncertainty resolves first, or in what sequence decisions will have to be made. This is what rules out a simple staged model.
Interdependencies among projects
Projects in the portfolio affect one another, so valuing each in isolation and summing produces the wrong number.
Transparency is vital
The method has to be inspectable by the people using it. This is why visualisation is one of its declared key concepts.
Naming the conditions does a limitations statement's job in advance. It tells you when the method applies and, by implication, when it does not — the distinction a good critique draws between a limit of applicability and a flaw.
The limitation profile is the familiar single-organisation, single-application one. Whether the method transfers is not established here, and the paper does not claim it is.
What the five have in common
FIVE CROSS-CUTTING OBSERVATIONS ABOUT THESE FIVE PAPERS
| # | Observation | What it means for your reading |
|---|---|---|
| 1 | Only one of five collects primary data from people. One is econometric on existing firm data, two are pure modelling, one is a single case study | "Research" in this literature very often means reasoning over existing material. Assuming you must go and ask someone is a habit, not a requirement |
| 2 | Three share a house style — an OVERVIEW abstract box plus a KEY CONCEPTS line before the body | That is the journal's requirement, not the authors' choice. The same forcing device as the structured abstract in this library's examined conference paper |
| 3 | None declares a research paradigm | This now holds across ten examined works — five theses and papers, and these five journal papers — while the teaching material states it should be clear what paradigm you are working within |
| 4 | Two report results that do not support the expected direction, and neither buries it | P1's statistically insignificant average effect; P5's conclusion running against patent protection. Both are reported in the abstract, not hidden in a discussion |
| 5 | Journal length forces omission. None has a methodology chapter in the thesis sense, and two have no empirical method at all | A reader trained on thesis structure will misread these as incomplete when they are entirely conventional for their venue |
These are observations about these five papers only. They are not published statistics about the field and must never be presented as rates.
Reading a journal paper when you were trained on theses
The fifth observation causes the most trouble. A thesis-trained reader opens a seven-page paper, finds no methodology chapter, no limitations section and no philosophical position, and concludes it is thin.
Usually it is not. It is compressed to a page budget the author did not set, on a template the journal imposed. Judging it against a thesis contents page measures the venue, not the work.
WHAT IS MISSING BECAUSE OF LENGTH, AND WHAT IS MISSING BECAUSE OF DESIGN
| Absent element | In these five papers it is usually… | How to tell the difference |
|---|---|---|
| A methodology chapter | A length casualty. The method is compressed into a section or a paragraph | Look for whether the method is stated somewhere, even briefly. Stated but not elaborated is compression |
| Any empirical method at all | A design decision, in P2 and P5. There is nothing to report because nothing was collected | The paper's claim is analytical. Asking it for a sample is asking the wrong question |
| A limitations section | Sometimes replaced by scope conditions stated up front, as in P4 | Search the introduction and the method for what the paper declines to claim |
| A paradigm or philosophical position | Absent in all ten works examined in this library | Not a length effect. Seven pages and 22 pages both omit it |
Questions to put to a journal design before you use it as a model
- Does the paper generate any new observation of the world, or does it reason over existing material?
- If it collects data, does the variable that sorts cases into groups come from the same source as the outcome measure?
- If it collects none, are the assumptions the argument rests on stated where you can find them?
- Does it report the result that did not go its way, and where — abstract, results, or buried in discussion?
- Does it state the conditions under which its contribution applies?
- Is the design one you could actually run under your own access, timeframe and word count?
What to carry forward
- Five designs in one subject domain, and only one collects primary data from people. Two collect none at all.
- P1 opens by naming the quantity it cannot observe. Building a design around an acknowledged counterfactual is the most transferable move in the set.
- P1 reports a statistically insignificant average effect alongside a large small-firm effect. All its figures are findings of one study in one economy in one decade.
- P3's groups are defined by respondents rating their own organisations, and its capability measure comes from the same respondents. That bounds the finding to what directors say about themselves.
- P4 names four conditions its method is for, which does a limitations statement's job in advance.
- None of the ten works examined in this library declares a research paradigm, while the teaching material says it should be clear which one you are in.
- Missing sections in a short paper are usually a venue constraint, not a defect. Check whether the method is stated before you conclude it is absent.
Frequently asked questions
Does one of five collecting primary data mean most research does not involve people?
No. These are five specific papers supplied alongside the teaching material, all in one subject domain, and they support no claim about the field. What the set does show is that designs which reason over existing data, or over stated assumptions, are ordinary and publishable — so the assumption that a study must involve fieldwork to count is a habit rather than a rule.
What is wrong with a self-rated grouping variable?
Nothing, until the outcome you compare across the groups comes from the same respondent. In P3 the organisations were classified as high-effectiveness by their own directors, and those same directors rated the organisations' capabilities. The two measures are not independent, so a correlation between them may reflect how a person answers a questionnaire rather than anything about the organisation.
Can I write a paper or thesis with no data at all?
Two of these five papers do exactly that, and both are published — one proposes a valuation method, the other builds a formal model and reasons from it. The requirement shifts rather than disappears: a non-empirical contribution has to state the assumptions it rests on and the conditions under which it holds. For an assessed master's project, check what your own institution permits before designing around it.
Why do three of the papers look structurally identical?
Because they were published in the same practitioner journal, whose house style requires an OVERVIEW box and a KEY CONCEPTS line before the body. That structure is the venue's requirement, not the authors' design decision, and reading it as an authorial choice will mislead you about how the work was conceived.
Should I report a result that does not support what I expected?
Yes, and two of these five demonstrate how. One reports a statistically insignificant average effect alongside a strong effect in a subgroup, distinguishing the two rather than letting the favourable figure stand for both. The other reaches a conclusion running against the arrangement it examines. Both put the unfavourable result in the abstract, which is the opposite of burying it.
References and source attribution
- Five journal papers supplied as exemplars alongside the teaching material, published between 2000 and 2011 in economics and technology management journals, covering R&D subsidy evaluation, R&D project valuation under hedging, R&D organisational effectiveness, R&D portfolio uncertainty and the efficiency of patents. Design observations only; referred to on this page as P1 to P5.
- Bryman, A. 2016, Social Research Methods, 5th ed., Oxford University Press, Oxford — cited in the supplied source; the standing reference for survey design and measurement validity.
- O'Leary, Z. 2017, The Essential Guide to Doing Your Research Project, 3rd ed., Sage Publications, London — one of the general research methods texts cited across the supplied source.
- Quinlan, C. 2011, Business Research Methods, 1st ed., Cengage Publishing — cited in the supplied source, and the origin of the methodology framework used across this library.
- The supplied teaching source: consolidated weekly teaching notes and slide material accompanying the five papers, including the activity brief that required them to be identified and critiqued.
Suggested questions for Ask KEVOS
- Which of these five designs is closest to what my own project could actually run?
- Check my survey design for a grouping variable that comes from the same respondent as my outcome measure.
- My contribution is a method rather than a finding. What do I have to state instead of a sample?
- Help me identify the counterfactual my study cannot observe, and how to write about it honestly.
- Is this short paper thin, or is it compressed by its journal's length limit?