Skip to main content
Operationalizing Decision Yield - Creating and Applying the Decision Performance Audit

Posted by Guest Blogger, Ian Turvill.

Yesterday, I described how you could - in theory - develop a set of questions to measure Decision Yield for any decision-making process.

Today, I'm going to describe the practical application of those questions in creating a complete "Decision Performance Audit" to assess your organization's performance along the dimensions of Decision Yield and to identify specific areas for improvement.

Define Your Scope:

As I mentioned yesterday, Decision Yield cannot be meaningfully measured at the level of an overall industry.  Instead, when building a Decision Performance Audit that relies on the Decision Yield metric, it should be built at a level that is sufficiently specific and focused to be compelling to the business decision maker.

There are many different ways to further divide up an industry.   For example:

  • By industry segments, e.g., in telecommunications, wireline, wireless, MVNO, vs. ISP
  • By nature of products delivered, e.g., in banking, credit cards vs. savings accounts
  • By nature of the customers served, e.g., in long-distance telephony, consumer vs. commercial
  • By functional area, e.g., in insurance, underwriting vs. claims
  • By process, particularly where that process is considered by some in an industry to be “broken”, or requires complex coordination across multiple parts of an organization, e.g., managing attrition among credit card customers

There is no right or wrong way to divide up an industry – it’s what makes sense, particularly given the degree of difference among the nature of the decisions to be made.

It may also be very valuable to think about driving an organization to consider enterprise-wide issues relating to a particular decision domain.  For example, a “Decision Performance Audit for Banking Fraud” appears to me to be a compelling concept.  It would probably consist of a high-level set of Decision Yield measures and accompanying, detailed sets of measures for individual worksteps.

Build Out Your Question Set

Let's imagine that we wanted to create a "Decision Performance Audit for Insurance".  But insurance is a BIG area, so let's break it down a bit further.   So, let's say a "Decision Performance Audit for Insurance Underwriting".

But even that is insufficient, since underwriting for personal lines is very different from that for commercial lines, and certainly from underwriting in life insurance.  So, a "Decision Performance Audit for Personal Lines Insurance Underwriting" is probably what's required. (Breaking it out further between auto and home policies might even be necessary, but probably only in limited circumstances.)

So, given that you've defined an appropriate scope, what do the questions look like?

(For more instructions, please refer back to my first article in this series: Operationalizing Decision Yield - Defining the questions.)

Here are four examples that might apply in the case of Personal Lines Underwriting.

Decision Performance Audit for Personal Lines Underwriting:

Sample Question 1: What types of rating/underwriting decision tools are used as the basis of the risk assessment?

  • Laggard:  Written policy manuals only
  • Follower: Common use of predictive models
  • Leader: Expert, pooled, and empirical Models

Relates to: Precision (High)

Sample Question 2: What proportion of the business you quote is issued at the same rate as originally underwritten?

  • Laggard: >85%
  • Follower: >95%
  • Leader: >99%

Relates to: Precision (Low) and Consistency (High)

Deliver the Decision Performance Audit

I would recommend delivering the audit in three stages:

Stage 1: In the pre-work stage, it is important to identify the functions that will be impacted by the results of the Audit and therefore be included as the study.  For example, in personal lines underwriting, it probably makes sense for representatives of the following functions to be identified and tapped to participate:

  • Executive
  • Actuarial and Underwriting Management and Staff
  • Information Technology
  • Marketing and Distribution Management
  • Agents (Direct, Captive, and Independent) and representatives of other distribution channels, such as internet aggregators

Each representative should be given a copy of a pre-work questionnaire to complete, on a “best efforts” basis, using knowledge that is either commonly known in the organization, or is available through standard reports or easily executed queries.  If information is not available, then it is entirely acceptable to indicate “Not Known” as the response to a specific question.

The person completing the survey can be told at a high level why their responses are necessary, and explain the importance of the responses to the upcoming workshop effort, but little further explanation should be provided at this point.

Representatives can consult within their own function, but should may not consult or confer with representatives from other functions.  It is very important each function answers the questionnaire independently, because the potential divergence in responses across functions can provide valuable insights about opportunities and challenges.  Each questionnaire should be tailored to present only the questions of relevance to the particular function receiving it.

After the questionnaires are returned, you should enter the responses into a unified database. Use the information to generate “standard views” of the responses, including:

  • Overall Score: Across the dimensions of Decision Yield
  • Comparisons: How company compares with “minimum competitive levels” and “market leading levels”
  • Reasons: What are the factors that contribute positively to a company’s rating along a particular dimension? What are the factors that contribute negatively to the ratings?
  • Important Functional Differences: What might different ratings along the same metrics reveal?
  • Simulations: How might the overall level of performance improve if the company makes improvements in specific areas?

Stage 2: In the workshop stage, share the survey results to provide a full view of the company's decision-making performance.

But before you do, introduce participants to the Decision Yield concept and its value in improving decision-making efficiency and effectiveness.   For example, you could talk about companies in your industry who have staked out a dominant position in one or more Decision Yield dimensions.

Who is the leader in precision?  In credit cards, it might be Capital One.  Who is the leader in cost?  In insurance, GEICO is likely to rise to the top.  And who are the leaders in consistency, agility, and speed.  These illustrations should help your audience understand the strategic value of the Decision Yield concept.

Next you should introduce the results of the pre-workshop questionnaire.  Present the overall results and competitive comparisons, and then break the results down further, for example, by dimension or by function.

Use the results to spur decision-making related discussion among the group.  Pose key questions:  Are these results surprising? Do they reflect the company’s existing strategy? Should the firm consider changing its strategy to exploit areas of strength and to shore up areas of weakness?

Also use the discussion to identify very tangible advances in decision-making performance: Where are there immediate low-cost opportunities for improvement? What larger scale improvements could be made?

Stage 3:  After the workshop, you can deliver a post-workshop report that summarizes findings. Use this as an opportunity to detail the specific initiatives that could be made to improve decision-making.  Lay out, if at all possible, an overall ranking of opportunities, based on factors such as ease of implementation, investment requirements, and expected payback.

And then, "Hey, presto!", you have the mandate to make your organization that is powered to the gills by Enterprise Decision Management.

If you are thinking of building a Decision Performance Audit, please contact me or James for assistance.  It's quite possible that we already have a whole set of questions lined up and ready to go!

related posts