Continuing the series on Adaptive Control:
- Why do you need Adaptive Control?
- What's a basic technology approach for Adaptive Control?
- What is Champion/Challenger? - This Post
- What is Experimental Design?
- How do I do Decision Analysis?
Champion/Challenger is an important concept. The idea is that you identify your current approach as a “Champion” – documenting the business rules and analytic models that together represent your best approach to a given decision. "Challenger" approaches are then developed. Each Challenger differs from the Champion in some measurable and defined way. Perhaps it has different business rules, perhaps it uses a different risk model, perhaps it is more aggressive about retaining customers. Each Challenger will therefore deliver different results from the Champion.
These results may be better or worse but only testing the approaches with real transactions, in a live environment, can really show. A Decision Service is therefore configured to push a small percentage of the transactions through each of the Challenger approaches while pushing the majority through the Champion. Results from the different approaches can be compared and measured over time. If a Challenger does better than the Champion, it can be made the new Champion and the process of identifying and testing new Challengers repeated to continually improve the decision.
The advantage to considering multiple approaches in parallel is shown below. If we have a current strategy and a target, but unknown, optimal strategy then our objective is to move towards it over time. With only a single current approach, we gain experience only in a limited area and so can only move towards the optimal approach within that area of experience. This means potentially many attempts at an approach before we reach the optimal one. When it may take months or years to see the full impact of an action, this may be too slow to be acceptable. In contrast, if a number of Challenger approaches are also defined you dramatically increase the decision space considered and so can move toward the optimal approach more quickly. This increased speed of improvement is enhanced by the more systematic approach to comparing approaches offered by Champion/Challenger.
adaptive control, business rules, champion/challenger, decision service, predictive analytics
