Fraudsters are continually adjusting their strategies to circumvent fraud detection systems. In my last few posts, I've been discussing how adaptive analytics are built to counter this problem. Because of this, the latest version of FICO™ Falcon® Fraud Manager leverages adaptive analytics. The adaptive model adjusts the base neural network Falcon score in response to real-time fraud tactics that were not present at the time of the neural network model training.
New FICO research shows that this approach—coupling adaptive models with traditional neural network fraud models—can greatly reduce performance degradation for the neural network model. Results from our research simulation are shown in the graphic below.
Let me break down the chart above for you.
• The blue curve (Model A) represents a FICO™ Falcon® neural net model built and evaluated on data from 4/2008 to 3/2009. Intuitively, it performs well because it is being tested on the same timeframe on which it was built.
• To simulate model degradation, the red curve (Model B) shows a Falcon model evaluated on the same 4/2008-3/2009 timeframe but built on data outside of this date range. As expected, it does not perform as well as the Falcon model trained on the 4/2008-3/2009 data (Model A). We see a 9% relative reduction in fraud account detection at a 10:1 account false positive ratio (AFPR).
• The black curve (Model C) shows performance on the same timeframe of the Falcon model from the red curve combined with adaptive analytics. The observed performance not only meets but slightly exceeds that of the Falcon model trained on the 4/2008-3/2009 data (Model A)—a 2.3% relative lift in account detection. Compared to the simulated degraded model without adaptive analytics in the red curve (Model B), we see a 12.5% relative lift.
These results are intriguing because model degradation is always an analytics challenge, particularly in the ever-changing world of fraud. The study points to possible situations where annual fraud model retrains may be less critical given that an adaptive model is able to adjust to the production environment and fraudster tactics. And it reinforces the value of constant innovation in fraud analytics—a sentiment that drives our own research efforts as we continuously enhance our Falcon fraud models and explore novel adaptive/self-learning technologies.
