FICO World Preview: Leveraging Big Data Analytics for Streaming Data

By Scott Zoldi

In my session at FICO World on leveraging Big Data analytics for streaming data, we will be discussing the concept of multi-layered self-calibrating (MLSC) analytics, FICO’s latest Big Data analytic technique for in-stream prediction and streaming transaction data. This technology leverages a multi-layered model architecture to obtain impressive model performance and robustness compared to traditional analytic techniques.

A major implication of Big Data is that analytics must rely less on persistent (historical) data and instead adjust dynamically real-time in the stream. This is particularly true where data is dynamic and changing, such as in the ever-changing world of fraud, as I discussed in a Banking Analytics Blog post. We also see other applications for the innovation across industries, such for customer purchase propensity in retail, network security in telecommunications, cyber security in internet/network applications, social network analytics, and energy assurance for the electrical grid, as just a few examples. Compared to a traditional neural net, the MLSC model:

• Is built to adapt in production, unlike neural nets where weights are fixed after initial training.
• Can be easily tuned to the needs of a specific market and is more robust to model degradation. This is due to the more flexible, adaptable design of its hidden layer nodes (each a self-calibrating mini-model).
• Doesn't need tuning as frequently because of its adaptive nature.
• Can be leveraged even where there is no data for model development to allow for in-stream anomaly detectors to help find unknown signatures/anomalies in huge streams of data which cannot be persisted.
• Requires much less data during model development and has more tolerance for low-quality data.

This last bullet point is an important one. It means we can tune a MLSC model using roughly a week to a month of data, compared to about 18 months of historical data for a neural network model. Indeed, the MLSC models have demonstrated strong performance for clients without large amounts of data and with lower-quality reporting. This makes it the ideal technology for emerging markets where there isn’t sufficient data to leverage neural networks.

I’ll discuss this exciting development in more detail at FICO World. You can register for the event at http://www.ficoworld.com/.