John Sandifer, Joe LaLuzerne and Ian Turvill presented on both the EDM methodology that Fair Isaac has assembled from its experience and the concept of Decision Yield. Joe went first walking through the methodology. He emphasized the need for upfront investigation of opportunities and threats before diving into the development of a decision management solution. In particular the overlap and integration of ideas and opportunities is very valuable as it allows more to be covered by projects. Although Joe covered the other phases, the focus in this session was in the identification and justification of decisioning projects. He identified three unique aspects:
- A focus on decisions and on decision automation and management.
These are often customer-facing but make sure to look for hidden decisions and check out this post on how to justify optimizing decisions. This focus also helps you focus on data needed for decision-making not just on capturing all the data. - Use of Decision Yield throughout.
As a measure for decisions, picking projects, estimating the value of improving decisions and showing how well it went. - Focus on data, analytics, rules as the key components of a decision service
Not on other aspects of systems development or analytic development
Joe went on to discuss the exit criteria for each stage.
John then came on and focused on the need for a new measure at the high volume/low incremental value level and mapped these decisions to the moment of truth in a customer-facing transaction. Decision Yield is the measure that allows you to do this. The five dimensions of Decision Yield are:
- Precision
How targeted, how accurate, how granular - Consistency
Across time, channel etc - Agility
Time and cost to change systems - Speed
Elapsed time - Cost
Total internal and third party costs for a decision
He then introduced the concept of a decision performance audit, based on decision-specific questions and illustrated this with a specific example question. He then talked about the need to apply weights to these questions and see which dimension they impact the most and how they impact all 5 dimensions. The weighted responses can be summed and normalized so that 100 is a perfect score. Finally this is used to clarify thinking on strategic priorities, for instance by assigning points to each dimension. There are many applications for this approach, for instance to track performance over time, to pick between options and to estimate payoff from a project. This left Ian to present some data on Decision Yield.
Ian began by plugging the blog. One of the handouts was a sample decision performance audit (which you can download here: Decision Yield Questionnaire (100.7K)) and Ian offered to calculate a Decision Yield for anyone who filled them out. Each question allowed for an answer of basic to intermediate to best practice.For example in terms of decision models for marketing the range was from simple rules/decision trees (basic) to advanced constraint-based optimization (best practice). He used a real survey, turned in at the conference, and produced a decision yield.
Ian made the point that the use of decision yield is to compare actual performance with their strategic intent - how aligned are they. The example was very focused on efficiency (speed and cost were more than 50% for instance) and agility, in contrast, was not important probably implying a fairly stable environment. Finally Ian showed what would happen if they improved a number of areas.
A summary of the main session slides is shown below and covers:
- An outline of the 7 steps of the methodology
- The challenge of measuring different kinds of decisions
- A Decision Yield radar graphic
- When Decision Yield is a good measure
- An description of a decision performance audit
- An illustrative question for a decision performance audit
