Risk

Approximate Value Functions

Scaling sequential decision-making to realistic projects by approximating the value function from features instead of tabulating every state.

Beliefs

Tracking a probability distribution over hidden states — Bayesian filtering, Kalman and particle filters — to reason about project status you cannot directly observe.

Collaborative Agents

Aligning independent decision-makers toward a shared objective — coordination problems, equilibrium selection, common knowledge, and the role of communication.

Controller Abstractions

Representing a POMDP policy as a compact finite-state controller — a small graph of modes and observation-driven transitions that's cheap to run and easy to audit.

Exact Belief-State Planning

The partially observable Markov decision process and its exact structure — planning over beliefs, alpha vectors, and why exact solutions don't scale.

Exact Solution Methods

The Markov decision process and its exact solutions — value and policy iteration — for optimal sequential decisions across a project lifecycle.

Exploration & Exploitation

Balancing acting on what you know against learning what you don't — bandits, optimism under uncertainty, and the pilot-versus-commit decision.

Imitation Learning

Learning a policy from expert demonstrations — behavioural cloning, the compounding-error pitfall, and inferring the expert's underlying objective.

Inference

Computing posterior risk given evidence — exact inference by elimination, and Monte Carlo estimation when the model is too tangled to solve exactly.

Model-Based Methods

Learning a model of the environment from experience and planning against it — a sample-efficient way to decide well while the dynamics are still unknown.

Model-Free Methods

Learning good actions directly from experience without ever building a model — temporal-difference learning and Q-learning for when a credible model is out of reach.

Multiagent Reasoning

Reasoning strategically about other decision-makers — normal-form games, Nash equilibrium, and the prisoner's dilemma applied to project counterparties.

Offline Belief-State Planning

Approximating a POMDP policy in advance — point-based value iteration and related methods that track value only at representative sampled beliefs.

Online Belief-State Planning

Deciding from the current belief at run time by looking ahead over actions and observations — POMCP and belief-lookahead search for larger partially observable problems.

Online Planning

Deciding on the fly by looking ahead from the current state — forward search and Monte Carlo tree search — instead of precomputing a full policy.

Parameter Learning

Fitting a risk model's probabilities from data — maximum likelihood versus Bayesian estimation, and why sparse project data demands priors.

Policy Gradient Estimation

Estimating the gradient of expected return with respect to policy parameters from sampled trajectories, and reducing its variance for a reliable signal.

Policy Gradient Optimization

Turning gradient estimates into stable policy improvements — natural gradients, trust regions, and clipped objectives that prevent catastrophic updates.

Policy Search

Searching directly in the space of parameterised decision rules — evaluating policies by simulation and optimising them without computing state values.

Policy Validation

Testing a decision policy before trusting it — rare-event simulation, robustness to model error, and adversarial stress-testing of tail behaviour.

Risk11 Aug 2026 · 4 min read

Representation

Encoding project uncertainty as probability distributions and Bayesian networks — a shared, auditable model of what drives risk.

Sequential Problems

Strategic decisions over time among adapting agents — Markov and repeated games, the shadow of the future, and multiagent learning under non-stationarity.

Risk11 Aug 2026 · 5 min read

Simple Decisions

Turning beliefs and preferences into a single best choice — expected utility, risk aversion, decision trees, and the value of information.

State Uncertainty

Strategic decisions with partial information held unevenly among agents — Bayesian games, decentralised POMDPs, and reasoning about others' private knowledge.

Structure Learning

Learning the graph of a risk model from data — scoring fit against complexity, searching structures, and why correlation can't always reveal direction.