There's a more direct route to a good decision rule. Don't compute values and derive a policy from them — write the policy down with a few dials, and turn the dials until it works.
Everything so far reasoned about the value of states and then acted greedily. Policy search sidesteps values entirely. You express a decision rule as a parameterised policy — a formula with adjustable settings, such as "if schedule variance exceeds θ₁, add resources; if open risk exposure exceeds θ₂, escalate" — and you search directly for the parameter values that make the policy perform best. When you can simulate a project but can't model it cleanly enough for value iteration, this is often the most practical path.
1Judging a policy by simulation
To search for good parameters you need to score any candidate. That's approximate policy evaluation: run the parameterised policy through many simulated projects and average the return it earns. The average is noisy — each simulation is one roll of the dice — but with enough runs it ranks candidates reliably. The whole method rests on being able to simulate outcomes, not on having tidy transition probabilities.
2Ways to search the dials
- Local search nudges the current parameters and keeps changes that improve the simulated return — simple hill-climbing over settings.
- Genetic algorithms keep a population of policies, combine and mutate the better ones, and let good settings propagate.
- The cross-entropy method maintains a probability distribution over settings, samples a batch, keeps the best-performing fraction, and re-centres the distribution on them — tightening toward good regions each round.
- Evolution strategies perturb the current parameters in many random directions, then step toward the perturbations that scored well — robust and easy to parallelise.
Policy search optimises the decision rule directly against simulated performance, never computing the value of a single state. If you can simulate your project and describe your response as adjustable settings, you can tune it without a model.
Write your risk-response strategy as a rule with explicit thresholds and dials, then let simulation tell you where to set them — rather than arguing over trigger levels in a meeting. Because it needs only the ability to simulate outcomes, policy search suits messy, real-world response logic that resists clean mathematical modelling. The output is directly usable: a tuned, transparent set of thresholds your team can read and follow.