Combine everything Part IV has established — you can't see the state, only a belief — with the sequential decisions of Part II, and you arrive at the most faithful model of project decision-making there is. It is also one of the hardest problems in the field.
The partially observable Markov decision process (POMDP) extends the MDP with one honest admission: you never observe the state directly. Added to the usual states, actions, transitions and rewards is an observation model — the probability of each possible signal given the true (hidden) situation. You act, the world changes unseen, you receive a noisy observation, you update your belief, and you decide again. That is precisely the loop of running a real project.
1A POMDP is an MDP over beliefs
The key move that makes POMDPs tractable to think about is this: although you can't see the state, your belief (Chapter 18) is something you always know. And it turns out the optimal decision depends only on your current belief — so a POMDP is really an MDP whose "states" are beliefs. The catch is severe: beliefs are continuous, so this belief-MDP has infinitely many states. You've traded an unobservable state for an observable but infinite one.
2The structure that saves you: alpha vectors
Remarkably, the optimal value function over beliefs has a clean shape — it is piecewise-linear and convex. It can be represented by a set of alpha vectors, each corresponding to a conditional plan: a complete "if you see this, do that" strategy. The value of any belief is simply the best of these plans at that belief, and which plan is best depends on where your belief sits. Convexity carries a meaning worth savouring: value is highest when you're certain (at the edges) and lowest when you're maximally unsure — uncertainty itself is costly.
3Why exact solutions don't scale
Exact methods compute this set of alpha vectors — enumerating conditional plans and pruning those that are never best. They are correct and they define what "optimal" means. But the number of conditional plans explodes as the planning horizon grows: each extra step branches over every action and every possible observation. Exact POMDP solving is therefore feasible only for very small problems. Its lasting value is conceptual — it names the true problem, reveals its structure, and sets the target that the approximate methods of the next two chapters chase.
The POMDP is the honest model of deciding without seeing the state, and its optimal policy is a library of conditional plans, each best for a certain range of belief. Exact solutions establish what optimal looks like; they simply cost too much to compute at any real scale.
Recognise that managing a project genuinely is a POMDP — you act on beliefs about a state you can't see, and the right response shifts with your confidence, not just your best guess. The structure carries a practical lesson even without solving the maths: your response should depend on how sure you are, and uncertainty itself is expensive, so reducing it (better observations, earlier signals) has real value. The next chapters make this computable; this one makes it honest.