Guides

Exploration & Exploitation

Balancing acting on what you know against learning what you don

Imitation Learning

Learning a policy from expert demonstrations — behavioural cloning, the compounding-error pitfall, and inferring the expert

Inference

Computing posterior risk given evidence — exact inference by elimination, and Monte Carlo estimation when the model is too tangled to solve exactly.

Model-Based Methods

Learning a model of the environment from experience and planning against it — a sample-efficient way to decide well while the dynamics are still unknown.

Model-Free Methods

Learning good actions directly from experience without ever building a model — temporal-difference learning and Q-learning for when a credible model is out of reach.

Multiagent Reasoning

Reasoning strategically about other decision-makers — normal-form games, Nash equilibrium, and the prisoner

Offline Belief-State Planning

Approximating a POMDP policy in advance — point-based value iteration and related methods that track value only at representative sampled beliefs.

Online Belief-State Planning

Deciding from the current belief at run time by looking ahead over actions and observations — POMCP and belief-lookahead search for larger partially observable problems.

Online Planning

Deciding on the fly by looking ahead from the current state — forward search and Monte Carlo tree search — instead of precomputing a full policy.

Parameter Learning

Fitting a risk model

Policy Gradient Estimation

Estimating the gradient of expected return with respect to policy parameters from sampled trajectories, and reducing its variance for a reliable signal.

Policy Gradient Optimization

Turning gradient estimates into stable policy improvements — natural gradients, trust regions, and clipped objectives that prevent catastrophic updates.

Policy Search

Searching directly in the space of parameterised decision rules — evaluating policies by simulation and optimising them without computing state values.

Policy Validation

Testing a decision policy before trusting it — rare-event simulation, robustness to model error, and adversarial stress-testing of tail behaviour.

Risk13 Aug 2026 · 9 min read

Representation

Encoding project uncertainty as probability distributions and Bayesian networks — a shared, auditable model of what drives risk.

Sequential Problems

Strategic decisions over time among adapting agents — Markov and repeated games, the shadow of the future, and multiagent learning under non-stationarity.

Risk13 Aug 2026 · 9 min read

Simple Decisions

Turning beliefs and preferences into a single best choice — expected utility, risk aversion, decision trees, and the value of information.

State Uncertainty

Strategic decisions with partial information held unevenly among agents — Bayesian games, decentralised POMDPs, and reasoning about others

Structure Learning

Learning the graph of a risk model from data — scoring fit against complexity, searching structures, and why correlation can