The methods so far produce policies that are powerful but demanding — they track full beliefs and reason over them at run time. There's a way to keep most of the benefit while handing your team something they can simply follow.
A finite-state controller represents a policy not as a belief calculation but as a small internal state machine. It has a handful of controller nodes — think of them as operating modes — each of which prescribes an action, and transitions between nodes that are driven by what you observe. Instead of maintaining a probability distribution over states, the controller keeps just a small internal memory (which mode it's in) and reacts to signals. It's a compact, executable abstraction of a POMDP policy.
1Fitting the controller
The nodes, their actions, and their observation-driven transitions can be optimised — chosen to maximise expected value for a controller of a given, small size. Policy-iteration and gradient-based techniques improve a controller, and bounding its number of nodes bounds its complexity. You deliberately restrict the policy to something compact, then find the best policy within that restriction. It won't match the full optimal POMDP policy — but it comes remarkably close for a fraction of the runtime cost, and with none of the belief-tracking machinery.
2The trade, and why it's often the right one
A controller gives up some optimality in exchange for three things a full belief-based policy struggles to offer: it is cheap to run (a lookup and a transition, no belief update), transparent (a diagram anyone can read), and auditable (its behaviour can be inspected and signed off in full). For anything that has to be operated by people and stand up to governance, those properties frequently outweigh the last increment of theoretical performance.
An optimal policy no one can run is worth less than a near-optimal one a team can follow and an auditor can inspect. A finite-state controller compresses belief-based decision-making into a small state machine of modes and observation-driven transitions.
Part IV confronted the hardest single-decision-maker problem — acting on beliefs about a state you can't see: tracking beliefs, the POMDP and its structure, offline and online belief planning, and compressing it all into an executable controller. But every part so far has assumed you are the only one deciding. Real projects are crowded with other decision-makers — clients, contractors, regulators, competitors — each pursuing their own aims. Part V closes the series with the strategic uncertainty that arises when others are deciding too.
When you turn a decision framework over to a team, favour a form they can actually operate: a small set of clearly-defined modes, each with a prescribed action, and unambiguous rules for moving between them based on what's observed. A legible, auditable playbook that people follow correctly will outperform a theoretically optimal policy they can't execute or a governance board can't scrutinise. Give up the last few percent of optimality to buy transparency and reliability — it is almost always a bargain.