Case studies as evidence: when one example is enough and when it is not

Businesses learn heavily from examples. A competitor’s successful product launch, a failed project in the company’s own history, a customer that cut its costs dramatically with a new process or a well-known corporate collapse all shape how managers think. These examples are case studies, whether formally written up or simply told as stories. They are powerful because they are concrete, memorable and rich in detail. They are also dangerous, because a single vivid example can be mistaken for a general rule, success stories hide the failures that did the same things, and hindsight makes every outcome look inevitable.

Used carefully, case studies are a legitimate and valuable form of evidence. They are especially good at explaining how and why things happen in real settings: how a project went off track, why a customer chose a competitor, how a new process was adopted or resisted. They allow researchers to examine many factors together, in context, over time. What they cannot do on their own is show how often something happens or how large its effect is across many organisations. Knowing the difference between these purposes is the key to using case studies well.

This article explains what case studies are good for, how to choose cases, how to design a case study, how to collect and combine evidence, how to test rival explanations, how to compare across cases, how to generalise appropriately and how to avoid common traps, including the confusion of promotional case studies with evidence. It is part of a series on research and evidence for business decisions.

What a case study is

A case study is an in-depth examination of a particular case, such as a project, customer, site, product launch, decision or event, within its real context. It usually draws on several sources of evidence: documents, records, interviews, observation and data. Its strength is depth and context; its focus is on understanding the case as a whole, including how different factors interacted over time.

What case studies are good for

Case studies suit questions such as:

• How did this happen? For example, how a project’s costs grew from approval to completion.
• Why did people act as they did? For example, why a customer group adopted a new ordering system quickly while another resisted.
• What happens in a new situation? For example, the first use of a technology in a particular setting.
• Does a general idea hold in practice? For example, whether a widely recommended method works under particular constraints.
• What can be learned from finished work? For example, post-project reviews.

They are poorly suited to questions about prevalence, such as what share of customers have a problem, or about average effects across many organisations, such as how much a method improves productivity in general.

Choosing cases

Which cases are studied determines what can be learned. Common choices include:

• Typical cases, which represent ordinary experience.
• Extreme or unusual cases, such as an exceptional success or failure, which reveal factors that are hard to see in ordinary cases.
• Deviant cases, which did not behave as expected, and which challenge assumptions.
• Critical cases, chosen because if an idea works there, it is likely to work elsewhere, or if it fails there, it is likely to fail elsewhere.
• Contrasting cases, such as similar projects with different outcomes, which help isolate the factors that made the difference.

Selecting only successes is a frequent and serious error. Studying successful businesses to learn what they did ignores the businesses that did the same things and failed, a form of survivorship bias. Including failures and contrasts gives much stronger evidence.

Designing a case study

A useful case study design defines:

• The question: what the study aims to explain.
• The case and its boundaries: what is included, such as which project, which period and which parties.
• Initial propositions: the explanations being considered, which guide what evidence to collect.
• Sources of evidence: documents, records, interviews, observation and data.
• The analysis approach: how evidence will be organised and compared.
• How findings will be reported, including confidentiality.

Being explicit about boundaries matters. A study of “why the project was late” that looks only at construction may miss causes in design or approval.

Collecting and combining evidence

Case studies gain credibility by combining sources, a form of triangulation:

• Documents and records: plans, contracts, emails, minutes, change logs, cost reports and schedules.
• Interviews: with people in different roles and with different perspectives.
• Observation: of the site, process or product, where still possible.
• Quantitative data: costs, times, defects, sales or other measures.

Where sources agree, confidence grows. Where they disagree, for example when interviewees recall events differently from what the records show, the disagreement is itself informative. Building a timeline of key events, decisions and changes, with the source of each entry, is one of the most useful analysis tools. Keeping a chain of evidence that links each conclusion to the evidence behind it allows others to check the reasoning.

Testing rival explanations

A case study is strongest when it considers alternative explanations and shows why the evidence favours one over the others. If a project’s cost overrun is attributed to poor estimating, the study should also examine scope changes, market price rises, contractor performance and site conditions, and show which explanation the evidence supports. Without this step, case studies tend to confirm the explanation the researcher started with.

Comparing across cases

Multiple case studies examine several cases using the same questions and methods, then compare them. Cross-case comparison can reveal patterns: factors present in all successful cases and absent in unsuccessful ones, or conditions under which a method works and fails. A simple comparison table, with cases as rows and key factors and outcomes as columns, makes patterns and exceptions visible.

Generalising from cases

Case studies do not support statistical generalisation, from a sample to a population. They support analytical generalisation: learning how a mechanism works and under what conditions, which can then be applied with judgement to other situations. The appropriate conclusion is not “this happens in 70% of projects” but “projects with these features are vulnerable to this kind of failure, through this mechanism”.

Hindsight and storytelling

Hindsight makes outcomes seem obvious and decisions seem clearly right or wrong. Participants reconstruct their memories to fit what happened, and researchers are tempted to tell tidy stories. Using contemporary documents, such as meeting minutes and emails written before the outcome was known, helps reconstruct what people knew and believed at the time. Judging decisions by the information available when they were made, rather than by their outcomes, is fairer and more useful.

Combining cases with numbers

Case studies and quantitative data strengthen each other. Numbers show how widespread a problem is and how large its effects are; cases show how it arises and what can be done. A business might find from its records that a fifth of jobs overrun their estimates, then use case studies of several overrunning and on-budget jobs to understand the mechanisms, and finally track whether changes reduce the overrun rate. Moving between the two kinds of evidence avoids both the abstraction of numbers without context and the anecdote of stories without scale.

Writing up a case study

A useful case study report is readable and traceable. It typically sets out the question, the case and its context, a timeline of key events, the evidence collected, the explanations considered and the conclusions, with lessons and recommendations. Quotes bring the case to life, but they should be representative rather than chosen only for impact. Separating description from interpretation, stating limitations and linking each conclusion to its evidence allow readers to judge the findings. A one-page summary for busy readers, with the full report available, makes the study more likely to be used.

Cases for training and induction

Well-written internal cases make excellent training material. Discussing a real project, incident or customer problem, with names removed where appropriate, helps new staff understand how the business works, what can go wrong and how good decisions are made under uncertainty. Asking learners what they would have done with the information available at the time, before revealing what happened, builds judgement and counters hindsight bias. A small library of such cases preserves lessons that would otherwise leave with experienced staff.

Promotional case studies

Many published case studies are marketing: a vendor’s story of a satisfied customer, a consultant’s account of a successful engagement. They may be accurate, but they are selected to show success, rarely report failures or costs and usually lack independent verification. They are useful for understanding what a product or service does and for finding references to contact, but they are weak evidence of how well it works in general. The questions in the article on reading research critically apply: who wrote it, what was compared, what is missing and could the results be checked?

Learning from your own projects

Some of the most valuable case studies are of a business’s own projects, jobs and incidents. Post-project reviews, incident investigations and reviews of lost tenders are case studies when done systematically. The reviews that ask why article explains how to run reviews that produce learning, and the what your projects know that your plan doesn’t article explains how to feed findings back into planning. Small, everyday cases are valuable too, as the small failures worth explaining article describes.

Confidentiality and permission

Case studies often involve customers, suppliers and staff. Internal case studies should protect individuals and focus on systems rather than blame. Any case study that identifies another organisation or person in published material needs their permission, and confidential information from contracts or other organisations must not be disclosed. Where permission is not available, cases can be anonymised, but care is needed because details can still identify organisations in small industries.

A worked example

This is an illustrative example. The business, projects and figures are hypothetical.

A fit-out contractor has completed three similar office refurbishments in a year. Two finished about 20% over budget; one finished within budget. Rather than blaming estimators, the operations manager runs a multiple case study.

For each project, the manager builds a timeline from the estimate, contract, variation register, site diaries and emails, and interviews the estimator, project manager and site supervisor. A comparison table records factors such as the completeness of drawings at tender, the number of variations, the timing of client decisions, subcontractor changes and site access issues.

The pattern is clear. The two overrunning projects were tendered on incomplete design drawings, and most of their overruns came from variations in the first month, when design details were finalised. Estimating rates were similar across all three. The rival explanation of poor subcontractor performance did not hold: subcontractor costs tracked the variations. The contractor changes its tender process to price incomplete designs with explicit allowances and assumptions, and to propose design-completion workshops before construction starts.

How this applies to a small Australian business

• Use case studies to explain how and why, not to estimate how often.
• Include failures and contrasts, not only successes.
• Define the case’s boundaries and the question clearly.
• Combine documents, interviews, observation and data.
• Build timelines from contemporary records.
• Test rival explanations before concluding.
• Compare several cases where possible.
• Combine cases with numbers: data for scale, cases for mechanisms.
• Build a library of internal cases for training and induction.
• Treat vendor case studies as illustrations, not proof.

Common mistakes

• Generalising from one vivid example.
• Studying only successful cases.
• Relying on memories reconstructed after the outcome.
• Ignoring evidence that contradicts the preferred story.
• Judging decisions by outcomes rather than the information available at the time.
• Choosing quotes for impact rather than because they are representative.
• Publishing identifiable details without permission.

Frequently asked questions

Can one case study prove anything? It can disprove a general claim, show that something is possible and explain how something happened in that case. It cannot show how common an outcome is.

How many cases are needed? It depends on the question. Several contrasting cases usually reveal much more than one, but the depth of each case matters more than the number.

Are internal project reviews case studies? They are when they define the question, gather evidence systematically from several sources and test explanations, rather than simply recording opinions.

How can hindsight bias be reduced? Use documents written at the time, ask people what they knew and expected then, and judge decisions against the information available when they were made.

Should case studies name the people involved? Internal reviews should focus on systems and decisions rather than individuals, and published cases should name people or organisations only with their permission.

Questions to ask

• What exactly are we trying to explain, and which cases will help most?
• Are we including failures and contrasting cases?
• What rival explanations should we test?
• Which contemporary records can we use instead of memory?
• Do our case write-ups link each conclusion to the evidence behind it?
• Which past projects or incidents would make useful training cases?
• How will we apply what we learn to future work?

Bringing it together

Case studies are powerful evidence for how and why things happen in real settings. Choose cases deliberately, including failures, contrasts and critical cases; define questions and boundaries; combine documents, interviews, observation and data; build timelines and chains of evidence; test rival explanations; compare across cases; and generalise to mechanisms and conditions rather than frequencies. Treat promotional stories as illustrations, and use your own projects as a steady source of evidence for better decisions.

---

Source: KEVOS editorial notes. Examples and figures in this article are illustrations.