(Posted by guest blogger, James Taylor)
As I talk to more companies adopting EDM (and more companies delivering it), it is increasingly clear that the role of adaptive control cannot be understated. Simply put, adaptive control is the systematic testing of alternative approaches to decision making on a continual basis. This generally means setting up a decision with a "Champion" approach and then randomly selecting a small percentage of those transactions to use a different "Challenger" approach instead. Thus most transactions get the approach expected to be the best but a few get alternatives. The explains why adaptive control is often called "Champion/Challenger".
Adaptive control allows both randomized testing to see which of several alternatives might work best when there is no way to know in advance while also allowing potential alternatives to be (reasonably) safely tried when results cannot be assessed without real transactions and real consequences. Regular and robust use of adaptive control also allows organizations to spot when existing approaches are losing their efficacy and to detect sudden or hidden changes in the environment that affect customer behavior.
While it is fairly common to see adaptive control in the analytic space, it is less clear to see it when rules are being applied. This makes sense, to some degree, because many rules-based decision automation projects are compliance-focused. Clearly if the rules are driven by regulations and policies, then trying out alternative approaches is counter-productive - only one approach can be "right" and there is no ambiguity. If, however, the rules are judgmental and based on how the organization thinks customers might respond, then adaptive control can make just as much sense as it does when trying alternative predictive analytic models.
Getting started with adaptive control need not be a terribly complicated process. The first, and perhaps the most problematic step, is to agree what it is that measures the success of a decision and over what timeframe. Is it just customer profitability or some combination of profitability and retention? Should retention be measured in months or years? How much is it worth to us to have a customer accept this particular set of terms and conditions for a product? Without agreement on this it will be impossible to meaningfully compare approaches so it is really important not to skip this step. Decide what matters to you and how you can measure it.
The next step is one of using an existing performance management environment or dashboard to track the effectiveness of the decision in the now defined terms. At this point you will always be using the same approach for every transaction/customer so no comparison will be possible but you will get an understanding of what measures matter and how they can be effectively displayed/tracked.
Once you now how well a decision is going, you can start to add adaptive control. This means doing several things in a coordinated way:
- Recording which approach is taken for each transaction/customer
- Adapting the reporting infrastructure to both show the overall results and allow comparison by approach within those results
- Adapting your decision service to choose randomly between multiple approaches and configuring it so that business owners can determine what percentage of the transactions will go through each approach. Any Business Rules Management System will handle this kind of change easily.
My advice would be to use the same approach initially for the champion and for how ever many challengers you want to be able to manage. This will allow everything to work and be tested without putting any transactions through a different approach - all the approaches will be the "champion". With all the infrastructure working you can start to design alternative approaches to act as challengers. This should start small - very minor differences from the champion - that perhaps move thresholds a little or challenge a long-held and oft-tested assumption. As you gain more confidence with the approach you can get more ambitious, though most challengers are not that different from the champion.
To try and explain the concepts visually, Rebecca Carrer developed a great flash animation. You can access it on her website at www.rhcvisualwriting.com - a list of examples appears on the right and the adaptive control movie is the first one. I really like the work Rebecca did on this and the animation is certainly easier to understand than static diagrams.
For more information on adaptive control, check out these links.
- Posts on the EDM blog
- My entry on Adaptive Control in the SmartEnoughWiki
- The more general wikipedia entry.