Book Review: Making Robust Decisions

I just finished reading Dr David Ullman's new book, "Making Robust Decisions". The book is a discussion of the challenges of making complex decisions, especially those with alternatives and uncertainty, and a methodology/software platform for approaching these kinds of decisions. It's a quick read with some great advice for anyone trying to make decisions, even if they don't want to adopt a more formal methodology. The sections on uncertainty, weighting of criteria and belief maps were particularly interesting. A couple of things stood out from an EDM perspective:

There was a great list of questions about a decision that could be applied to a decision you were automating as easily as they can be to a more manual one:

1. Which is the best alternative?
2. What is the risk that our decision will not turn out as we expect?
3. Do we know enough to make a good decision yet?
4. Is there buy-in for the decision?
5. What do we need to do next to feel confident about our decision, within the scope of our limited resources?

David also had a great definition of the phrase "robust decision"

"A robust decision is the best possible choice, one found by eliminating all the uncertainty possible within available resources, and then choosing with known and acceptable levels of satisfaction and risk"

Again, a definition that would work just as well as an objective for an automated decision, though it might be usefully rephrased this way for decision automation:

"A robust decision service is one that makes the best possible choice each time it is called, one found using rules and analytics that eliminate all the uncertainty possible within available resources, and then choose with known and acceptable levels of satisfaction and risk from the available options"

The book had some nice discussions of decision-making processes - I liked the "Ben Franklin" method. This involves making a list of Pros and Cons, estimating the importance of each, eliminating items from the pros and cons lists of roughly equal importance (or groups of items that can be traded off against each other) until one column (pro or con) is dominant. The basic decision-making structure of Observe-Orient-Decide-Act was well described as was Omphaloskepsis, another of my favorite approaches. The discussion of the impact of not deciding (management by wringing of hands) is well covered, David noting that in business the competition keeps taking action and you keep using resources without adding the value a decision would add. In other words, getting stuck at the decision point can have severe, even grave consequences. He also discussed briefly how a well defined decision process can help you benefit from the wisdom of crowds in that groups of estimates are better than single ones. I think the concept of Decision Yield could also play well here as it takes a nicely multi-dimensional approach to considering "rightness" of a decision.

David also listed a number of problems with human cognition that are worth considering. Humans:

• Are poor at accounting for uncertainty
  Some mathematical models are much better at it
• Suffer from anchoring or setting a biased context for decisions
  While you can build this bias into an automated decision, computer systems do not inherently suffer from anchoring and in particular are good at avoiding being anchored on a subset of the available facts
• Have a conflict between needs and accuracy - a single value is desired when a realistic assessment would have a range
  This is a problem for automated decisions too - there can be a pressure to have a yes/no answer when a sliding scale one would be better.
• Perceive high uncertainty as a lack of knowledge making people unwilling to state uncertainty
  Coding uncertainty into an automated decision may, paradoxically, be easier that admitting to it as people will accept that the computer needs to know the uncertainty
• Are optimistic in hindsight leading to poor estimates
  Analytic models based on "hindsight" are neither optimistic nor pessimistic - they just are.

I'll end with David's definition of decision management which can clearly be applied to decision automation in an EDM context.

Decision management is determining what-to-do-next with the available information in order to make the most robust decision as a part of standard work processes, and documenting the results for distribution and reuse

You can buy the book here directly or from Amazon here. You might also enjoy Blink (reviewed here).

Technorati Tags: analytic application, analytics, decision automation, decision service, EDM, Enterprise Decision Management, predictive analytics, risk management, statistical analysis, robust decision