Why This Matters: The Most Valuable Risk Data You Will Ever Collect
There is a bitter irony in project management: the moment when the organisation knows the most about a project's risks is the moment when that knowledge is least likely to be captured. The project is closing. The team is dispersing. People are already mentally assigned to their next programme. The pressure to "move on" is immense.
Yet the insights locked inside the heads of project team members at close-out — what risks materialised, which ones were missed, which contingency plans worked, which controls failed, and what the team would do differently — represent the highest-value risk intelligence the organisation will ever possess. Unlike theoretical risk frameworks taught in classrooms, this data comes from direct experience under real conditions with real constraints.
If these insights are not captured, the next project team will reinvent the wheel. They will make the same mistakes, miss the same risks, and rediscover the same lessons — at the same cost.
This article examines how to conduct an effective risk lessons learned session, how to transition from lessons learned to a formal project risk audit, and how the outputs of both feed back into organisational systems, training, and policy to prevent the repetition of failure.
What Is the Difference Between Lessons Learned and a Project Audit?
These two activities are frequently confused, but they serve fundamentally different purposes:
| Dimension | Lessons Learned | Project Risk Audit |
|---|---|---|
| Who conducts it? | The project team themselves | An independent external team not involved in the project |
| Perspective | Internal, experiential — "what did we learn?" | External, evaluative — "what does the evidence show?" |
| Timing | Immediately after project close-out (while memories are fresh) | After lessons learned are documented (builds on their output) |
| Method | Facilitated team discussion, brainstorming, debrief | Document review, evidence gathering, analysis against objectives |
| Focus | What went right, what went wrong, what would we do differently | Did the project produce what it intended, and how effectively? |
| Output | List of insights, recommendations, and corrective action proposals | Formal audit report with findings, root causes, and institutional recommendations |
| Analogy | A military debrief — capturing the intensity of recent combat experience | A military after-action review — systematic evaluation by external observers |
The flow is sequential and additive: lessons learned feed the audit, and the audit feeds institutional change. Skipping lessons learned and going straight to audit wastes the experiential insights of the team. Doing lessons learned without an audit misses the systematic, evidence-based evaluation that connects project experience to organisational improvement.
How to Conduct a Risk Lessons Learned Session
Timing and Participants
A lessons learned review should be conducted soon after project close-out — ideally within two weeks, before team members are reassigned and their memories fade or are overwritten by new project concerns.
The session should include:
- All project team members (not just the PM and senior leads)
- A member of top management (to signal that lessons will be acted upon)
- Stakeholder representatives (to capture external perspectives)
- A skilled facilitator (ideally someone not from the project team, to prevent groupthink and ensure honest discussion)
The Two-Question Structure
The supplied project-risk guidance recommends structuring the session around two deceptively simple questions:
Question 1: What went right? (Things we want to recreate for future projects)
This question captures the positive risk management practices — the things that worked and should be institutionalised. It is asked first because it sets a constructive tone and prevents the session from becoming a blame exercise. Question 2: What went wrong? (Things we want to correct for future projects)
This question captures the risk management failures — the things that did not work and need to be addressed. The focus should be on dimensioning what went wrong in terms of unanticipated or unmitigated risks and uncertainty, not on assigning blame to individuals.
Lessons Learned in Practice — The Avionics Case Study
The supplied project-risk guidance documents a lessons learned session from a real avionics instrument product development project that illustrates the depth and candour achievable when the process is done well. The case is instructive because the project ultimately succeeded — but the lessons learned reveal how close it came to failure and what systemic issues must be addressed for future projects.
What went right:
The team identified several factors that contributed to project success:
- Focused, largely full-time team assignments — avoiding the productivity loss of multitasking across programmes
- Team autonomy — the team drew clear boundaries with the customer on appropriate windows for changes and disruptions
- Minimal scope creep — the team controlled scope effectively
- Rapid resource allocation — test equipment and other resources were allocated quickly to resolve problems
- Contingency plans in place — when needed, pre-planned contingencies were available
- Strong intra-team communication — team members were responsive to each other
- Consistent top management priority — the project was high-visibility from the beginning and maintained corporate support throughout
- Technically feasible scope — the project was challenging but not insurmountable
- Clear distinction between controllable and uncontrollable factors — the team managed accordingly, particularly around flight test dependencies
- Flexible programme manager — open to change and task-oriented in meetings rather than process-oriented What went wrong:
The team identified significant systemic issues:
- Procedure conflicts — company process and procedure requirements and differing interpretations of documentation requirements sometimes acted as barriers to necessary work
- Documentation followed verification — procedures were described after completion rather than guiding it, creating compliance risk
- ISO application confusion — uncertainty in applying formal ISO procedures versus "actual" procedures the team deemed necessary to deliver on time
- Corporate restructuring impacts — a company split during the project created resource issues, forcing ad-hoc approaches to engineering change notices and assembly drawings
- Document numbering system failures — created significant tension and uncertainty
- Training gaps — staff involved in production tasks did not always have adequate guidance or training
- Ineffective version control — some documents and configurations lacked disciplined versioning
- Stress from long hours — unsustainable working patterns affected team performance and morale
- Incomplete big-picture communication — team members sometimes lacked context for how their work contributed to the whole project
- Meeting inefficiency — despite agendas, meetings sometimes devolved into unproductive venting
- Military-to-commercial transition friction — residual cultural conflicts between military documentation standards and commercial approaches
From Insights to Corrective Actions
The lessons learned session does not stop at identifying what went right and wrong. It must produce specific, actionable corrective recommendations that can be evaluated for implementation. The avionics case produced the following corrective actions:
| Issue | Recommendation |
|---|---|
| Document numbering system dysfunction | Set aside dedicated time to develop a new numbering scheme reflecting current business practices |
| Gap between documented and actual procedures | Document how procedures were actually followed for ISO audit preparation; review process sequence and review steps |
| No accountability for master charting | Assign clear accountability for the master chart function |
| Inadequate documentation tools | Evaluate and acquire appropriate tools for process documentation and version control |
| Ineffective document revision management | Develop a referenced master list of documents and acronyms for revision tracking |
| Staff non-compliance with procedures | Develop accountability mechanisms for following established procedures, with consequences |
| Late equipment procurement | Build required test equipment requirements into the scheduling process early; anticipate asset issues proactively |
| Failure to capture issues in real-time | Establish a running mechanism to identify and capture project issues, processes, and corrective actions as they occur |
| Product development vs manufacturing conflicts | Acknowledged but tabled for further organisational discussion |
| Concern that identified issues will not be addressed | Present findings to top management with requests for specific executive commitments on corrective action |
The Project Risk Audit
Purpose and Scope
The project audit starts with the appointment of an auditor — typically another project manager with experience in project planning and control — and an audit team of objective outsiders who were not part of the project.
Unlike the lessons learned session, which focuses on team experience and feelings, the audit is focused on independent gathering of information and documents from the project, and reviewing them against the goals, objectives, and best practice criteria.
The fundamental question of the audit is: "Did the project produce what it intended to produce, and how effectively and efficiently?"
The 10-Point Risk Audit Checklist
The supplied project-risk guidance identifies ten key aspects that a project risk audit should address:
| # | Audit Focus Area | Key Question |
|---|---|---|
| 1 | Business Planning | Were the risks that actually occurred identified adequately in the early business planning process? |
| 2 | Follow-up Response | Were those same risks monitored and controlled throughout the project? |
| 3 | Organisation-wide Culture | Did top management support effective risk management as part of the project cycle? |
| 4 | Project Team | Did project team members perform their roles as "risk managers" and integrate risk into their daily work? |
| 5 | Risk Identification, Assessment & Response | Was there a systematic process to identify, assess, and respond to risks? |
| 6 | Key Processes, Decisions & Milestones | Were key risk-related processes — testing, reliability, QA, decision trees, integration milestones — actually followed? |
| 7 | Resources | Did the project experience resource constraints, and were constraints managed with buffers? |
| 8 | Safety & Reliability | Were the correct tests and reliability processes in place? |
| 9 | Risk-based Scheduling | Were project schedules adjusted using risk inputs and PERT analysis? |
| 10 | Monitoring | Were risks followed in the project process and were decisions made on mitigation and contingency that reduced risk? |
The checklist operates at three levels — strategic (did the organisation set the conditions for effective risk management?), process (were the right risk management processes in place?), and execution (were the processes actually followed and were they effective?).
Conducting the Audit
The audit process follows a structured methodology:
Step 1 — Scope Definition. Define which aspects of the project's risk management will be audited, based on the lessons learned output and top management priorities. Step 2 — Document Collection. Gather all project risk documentation: risk management plans, risk registers (all versions), risk matrix, treatment plans, meeting minutes from risk reviews, change logs, earned value reports, and project review presentations. Step 3 — Evidence Analysis. Compare documented risk management activities against the 10-point checklist. Look for gaps between what was planned and what was executed, between what was identified and what actually occurred, and between what was recommended and what was implemented. Step 4 — Stakeholder Interviews. Interview key project stakeholders, including the PM, team leads, functional managers, the customer, and top management, to validate documentary findings and capture perspectives not reflected in formal records. Step 5 — Finding Development. Develop audit findings that connect observed issues to root causes and quantify impacts where possible. Each finding should be structured as:
- Observation: What happened (or did not happen)
- Root Cause: Why it happened
- Impact: What effect it had on the project
- Recommendation: What should change for future projects Step 6 — Report and Presentation. Present audit findings to top management with specific, actionable recommendations for institutional change — policy revisions, procedure updates, training programme modifications, tool acquisitions, or organisational restructuring.
The Defence and Heavy Engineering Context
The Unique Value of Defence Lessons Learned
In defence and heavy engineering, the stakes of failing to capture and act on risk lessons learned are amplified by several factors:
Long programme lifecycles mean decades between similar projects. If Australia builds a class of submarines every 25 years, the workforce that built the previous class will have retired by the time the next programme begins. Without institutionalised lessons learned, the knowledge walks out the door. Classification constraints limit informal knowledge transfer. In classified programmes, you cannot simply write a blog post about what you learned. Lessons learned must be captured in appropriately classified documentation and managed through information security frameworks — adding complexity but not diminishing the imperative. Complex supply chains distribute lessons across organisations. On a defence programme with 200+ subcontractors, risk lessons exist throughout the supply chain. A robust lessons learned process must capture insights from key subcontractors, not just the prime contractor. Regulatory and governance requirements mandate audit. In a national defence authority procurement, project completion reviews and post-implementation reviews are mandated under the public-sector acquisition framework. These are not optional — they are governance requirements with defined reporting structures.
The Postmortem Culture Problem
The supplied project-risk guidance describes the project audit as "the process of coming down off the mountain after the battle to kill the wounded." This vivid metaphor captures a genuine cultural problem: many organisations treat audits as adversarial exercises designed to find fault rather than improvement opportunities.
Building a constructive audit culture requires:
- Separating the audit from performance evaluation — audit findings should inform process improvement, not individual performance ratings
- Focusing on systems, not people — root cause analysis should trace failures to organisational systems and processes, not to individual blame
- Demonstrating follow-through — the organisation must visibly implement audit recommendations, or the audit process loses credibility
- Resourcing the audit properly — audits conducted under time pressure with insufficient access to documents and people will produce superficial findings
From Lessons to Institutional Change
The final — and most critical — step in the risk lessons learned and audit cycle is translating findings into institutional change. This means embedding lessons into:
| Change Target | Mechanism |
|---|---|
| Policies | Update risk management policies to address gaps identified in the audit |
| Procedures | Revise project procedures to incorporate new risk management steps or controls |
| Templates | Update risk register templates, risk matrix formats, and treatment plan structures |
| Training | Incorporate case studies from real project risk experiences into training curricula |
| Tools | Acquire or configure tools to address capability gaps (e.g., version control, document management) |
| Organisational structure | Restructure roles and responsibilities where the audit reveals accountability gaps |
Without this final step, the entire lessons learned and audit process is an intellectual exercise. The goal is not to produce a report. The goal is to change how the organisation works so that the risks of the last project do not recur in the next one.
Common Pitfalls in Lessons Learned and Audits
Pitfall 1 — Lessons Identified, Not Lessons Learned
The most widespread failure: lessons are identified in the session, documented in a report, filed in a project archive — and never acted upon. Unless the organisation implements the recommendations, nothing has been "learned." It has only been observed. A lesson is not learned until it changes behaviour.
Pitfall 2 — Blame-Focused Sessions
If the lessons learned session devolves into finger-pointing, team members will stop contributing honestly. Future sessions will produce superficial, politically safe observations rather than the candid insights that drive genuine improvement. The facilitator must maintain a "no blame" environment focused on systems and processes, not individuals.
Pitfall 3 — Timing Too Late
Conducting lessons learned six months after project close-out, when the team has dispersed and memories have faded, produces significantly less valuable output than a session held within two weeks. Timeliness is not optional.
Pitfall 4 — Audit Without Independence
An audit conducted by team members who were part of the project lacks the objectivity needed to identify systemic issues. The auditor must be an outsider — someone who can examine the evidence without the emotional investment of having lived through the project.
Pitfall 5 — Ignoring What Went Right
Lessons learned sessions that focus exclusively on failures miss the opportunity to institutionalise success. Identifying and replicating the practices that worked — the focused team structure, the autonomous decision-making, the early contingency planning — is as valuable as correcting the practices that failed.
Key Takeaways
Lessons learned and project audits serve different but complementary purposes. Lessons learned capture the team's experiential insights; audits provide independent, evidence-based evaluation.
Timing is critical. Lessons learned sessions must be conducted within two weeks of project close-out while team members' memories are fresh and detailed.
The two-question structure works: "What went right?" and "What went wrong?" — asked in that order — create the foundation for constructive, honest reflection.
The 10-point audit checklist spans strategic, process, and execution levels, ensuring that the audit addresses both the organisational conditions for risk management and the actual execution of risk processes.
Lessons identified are not lessons learned. Unless recommendations are implemented and institutionalised — through policy changes, procedure updates, training programmes, and tool improvements — the cycle of repeated failure continues.
Top management commitment is the single most important success factor. If the organisation has a history of ignoring lessons learned recommendations, the process loses credibility and future teams will not invest genuine effort.
Defence contexts demand formalised capture and classified management of risk lessons, given long programme lifecycles, personnel turnover, and information security requirements.