David Raab has a new article in DM Review on "New Trends in Predictive Analytics". David correctly identifies that many of the techniques using in predictive analytics, or data mining, are not that new (there are some like the genetic algorithms Fair Isaac and others are working on) but that the ability to really use these is new. David identifies several key areas where recent improvements have been made:
- Real-time execution
- Front-line deployment (including SOA)
- Customer-level decisions
But David goes on to document some interesting challenges. Firstly he points out that widespread use of analytics means that both that large numbers of models are needed creating challenges in terms of rapid development and deployment of models. Secondly the use of models in operational processes can mean that the users of those processes (call center representatives or the website) are not sophisticated enough to understand when a model is going wrong. Thirdly there is an issue as more and more models are developed - how to manage the interaction of those models.
These are all great points. There are a number of strategies for handling the number of models required. Some companies are developing better, more automated analytics engines. Some, like Fair Isaac, are developing tools and algorithms to drastically reduce the number of manual steps required by an expert modeler so that one modeler can produce more, and more accurate models. Combined with tools, like Model Builder, that automate the deployment of models into production these approaches can make the number of models required manageable. The use of an Enterprise Decision Management approach can also help. EDM links the interaction rules with the models that are part of the interaction to automate decisions making the decisions easier to understand and manage. It also gives business users some control over the way the model is used without them having to understand how the model was built.
The problem of handling the interactions of large numbers of models and the tradeoffs between them requires new approaches such as decision models at can optimize rules given the tradeoffs between various models.