On this blog, I regularly share information about the latest advances in fraud analytics. But our clients are not looking for innovation purely for innovation’s sake. An advance is only worth its salt if it measurably improves fraud detection.
Recently, my team tested the performance of two new fraud analytics that I’ve blogged about before—specifically Behavior Sorted Lists and Adaptive Analytics. In this post, I'll quickly recap each innovation and share those latest performance results.
Behavior Sorted List technology identifies cardholder "favorites"—or recurrences—over the transaction streams. These might include favorite ATMs that are close to work or home, favorite gas stations along a daily commute, and preferred stores for internet shopping. Behavior Sorted Lists can distinguish between frequently repeated transactions that indicate normal spending (what we data scientists call “in-pattern” transaction activity) and infrequent activity that is far more likely to be fraudulent (“out-of-pattern” activity). This ability enables faster fraud detection with lower false positive rates—that is, fewer declines on legitimate transactions.
The graphics below compare performance of FICO® Falcon® Fraud Manager 6 with and without Behavior Sorted Lists, using the most recent International Credit Models (ICM) 12. We observed substantial improvements using Behavior Sorted Lists, looking at both account detection rate and real-time value detection rate. I’ve highlighted improvements at a few account false positive ratios.
Digging deeper, we analyzed performance on a number of fraud types, including cross-border and card-not-present (CNP) fraud transactions. The following two plots show that the ICM 12 model with Behavior Sorted Lists outperforms the base model. It significantly improves transaction detection rates over a range of non-fraud transaction review rates.
We also evaluated the performance lift of Adaptive Analytics. Adaptive models work in conjunction with neural network fraud models, continually adapting the neural nets based on ever-changing fraud patterns that emerge over time. This not only improves model performance, but also extends the useful lifetime of static neural network models.
This ability to reduce model degradation is especially helpful in emerging international markets where we observe higher market dynamics. That’s why we tested our latest International Credit Models on not only in-time data (that is, the development data), but also on out-of-time data (outside the development data range). This evaluates whether the model will maintain robustness when deployed in a more dynamic production environment.
Performance results are shown in the graphics below. “AA” stands for Adaptive Analytics. The green curve represents the base ICM 12 performance—without Adaptive Analytics—on the in-time data that the model was built on. The red curve shows performance of the same model evaluated on out-of-time data. As you would expect, we see some performance degradation compared to the green curve. The blue curve shows the performance of the adaptive model on the same out-of-time data.
The adaptive model clearly outperforms the base model on the out-of-time data. Compared to the in-time performance of the base model, the adaptive model’s out-of-time performance is close for account detection rate, and approximately the same level for real-time value detection rate. It’s strong evidence that using Adaptive Analytics is highly effective in preventing model degradation.
Clients of our International Credit Models can take advantage of both Both Behavior Sorted Lists and Adaptive Analytics with the release of v12 later this year. ICM 12 has been trained on a more comprehensive pool of international credit consortium data, and it is capable of identifying a wide spectrum of credit fraud patterns across continents. These improvements will allow our clients to keep up with the latest fraud schemes in their respective regions.
