In a recent blog entry, Ted Kemp highlights an article on the basics of data mining. One question I get often is what is the difference between data mining and predictive analytics? Are they different or are they somewhere along a continuum? The article he mentions has an interesting quote:
At one end of the spectrum, a customer-profiling data mining model that runs once a quarter may only involve the data miner and ETL developer. At the other end of the spectrum, making online recommendations will require applications developers and production systems folks to be involved, which is usually a big deal.
So Warren Thornthwaite clearly takes the view that there's a spectrum because what he calls "a big deal" here is what I would call predictive analytics (likelihood of offer acceptance) embedded in a decision service (recommendation engine) that includes both the analytics and business rules about how and when to apply it. Thus rules and analytics or EDM.
However there is some danger in this "spectrum" approach. Warren goes on to say:
It's best to roll out the data mining model in phases, starting with a test version, to make sure the data mining server doesn't affect the transaction process.
And here is where I start to have to differ and why I think it is sometimes more useful to think of predictive analytics, or operational / executable analytics, as something separate. I believe that the set of technologies needed to inject analytic insight into a transactional process are different from those required to do data mining and data visualization etc.
- Predictive analytic models typically predict a likelihood of accepting a product offer say, for an individual customer based on the past behavior of all customers. While some of the same analytic techniques are used as in data mining, to see which segments prefer which products for instance, the result is an executable model or equation. You don't want to have to manually code the implications of a data mining report nor do you want to run a data mining server during a transaction. Transactional precision requires a different approach to reporting precision.
- Secondly you need to be able to wrap business rules around these analytics to make sure you apply common sense, context, regulations and policies. Embedding analytic insight into traditional code will likely result in a precise, but not very agile system. Business agility is too important to lose just to add analytic insight.
- Lastly you need to think about other places you might want the recommendation. It does no good to embed the recommendation decision into your online environment and then find you cannot reuse it in a batch letter printing process or your call center. Consistency of decisions is required across your enterprise.
An EDM approach is about getting precision, consistency and agility. Data Mining shares many useful techniques with EDM but EDM is more than just data mining on steroids.