Which works better for modeling credit risk: traditional scorecards or artificial intelligence and machine learning?
Given the excitement around AI today, this question is inevitable. It’s also a bit silly. While some new market entrants may have a vested interest in pushing AI solutions, the fact is that traditional scorecard methods and AI bring different advantages to credit risk modeling — if you know how to use them together.
Take, for example, our new credit decisioning solution, FICO Origination Manager Essentials – Small Business. It’s designed to help lenders make faster origination decisions without increasing risk. This new FICO product combines our well-established scorecard technology with AI to build better credit risk models.
How FICO Uses AI to Build Better Credit Risk Models
As with our other origination products, Origination Manager Essentials includes credit risk models, and these models are segmented — different types of small business customers and different credit products require different models to assess their credit risk.
In traditional risk modeling, customer segmentation is based on “hard” lines and broad categories, such as new customer vs. existing customer. This doesn’t capture the behavior of certain individual entities or more optimal ways to segment scoring models.
To build the models in Origination Manager Essentials, our data scientists used AI and machine learning algorithms to discover a better way to segment the scorecards. This allows us to apply AI to improve risk prediction without creating “black box” models that don’t give risk managers, customers and regulators the required insights into why individuals score the way they do.
We are now starting to use techniques such as collaborative profiles to reveal entity segmentation based on customer behaviors. We can then group customers into micro-segments based on that similarity, instead of typical segmentation approaches that rely on hard business attributes. For example, collaborative profiles derive behavioral archetype distributions — these could include archetypes that point to credit seekers building credit histories vs. those who have higher risk and covering misuse of credit elsewhere in their history.
The way that we can capture these subtle changes in behavior, and can incorporate them into the credit risk model, presents a distinct advantage for FICO customers. Our approach builds on mature, time-tested analytic models and scorecards, enhancing them with advanced AI technology to drive better segments and feature creation in models.
Another approach is to use AI and machine learning to “train” models to discover maximum predictive power, and find new relationships amongst input features that could produce a stronger model. For example, utilization is always an important feature in a credit model, as is delinquency, but a nonlinear combination of these can produce more optimal results in a machine learning model. You can then drive these new inputs into a traditional scorecard model to ensure explainability.
Improving Results with AI and Machine Learning
The two examples below illustrate how you can achieve better performance and explainability by combining machine learning and scorecard approaches.
When developing a credit card churn model, FICO data scientists used machine learning to discover a powerful interaction between recency and frequency of card usage. The option to include this interaction as a nonlinear input feature in an interpretable fashion into a scorecard led to a substantial improvement (~10%) of the lift measure, used to characterize the performance of attrition models. In addition, another 15% performance improvement was found by applying machine learning with a much larger set of features relating to event-specific recencies and frequencies. These predictive improvements in turn can translate into substantial portfolio profit gains for a much more precisely targeted retention strategy.
In a project to build a home equity portfolio with limited data, the lack of enough “bads” (poor performing loans) in our sample was causing some problems. By building a machine learning score with optimized hyperparameters, we were able to confirm that we were losing a significant amount of signal with a traditional scorecard. Using machine learning led us to change the model performance outcome from a binary outcome to a continuous outcome. By combining this technology with scorecard technology, we created a strong, robust, palatable solution and saw a 20% improvement in model performance (KS) over a traditional scorecard model alone (see below).
A Smarter Way to Use AI
FICO has long been involved with using AI as part of our analytic approach. How long? We recently filed a new explainable AI patent application to improve upon the IP of a FICO explainable AI patent granted in 1998 —that patent already expired.
Years of market experience have validated our approach, which is very different than the “move fast and break things” brashness we see from some AI start-ups in the originations space. They’re focused on using many types of alternative data, such as information gleaned from social media, to deduce credit risk.
Innovation is great, but you don’t want to naively throw in a lot of new data sources ––many of which may not be permissible in credit decision-making, and might be easily manipulated (like social media data) –– into an AI model that comes up with a score that may not be explainable. Why not? First of all, lenders in many markets do need to be able to explain how a customer was scored. Second, you don’t really understand what relationships are being learned from this data, and if these relationships really matter. By contrast, FICO analyzes new data sets along five lines to see if they will add value to credit risk scoring.
That message was part of my talk at the LendIt USA conference a few weeks ago, where I participated in the panel, “The New Frontier: AI, Machine Learning and Advanced Analytics.” You can see the video of the LendIt panel session here.
The Power of Scorecards
Scorecards are a powerful tool because, like AI, you can incorporate non-linearity in the input layer, and you can take advantage of different features that may be predictive in different ways for different subpopulations, by using segmented scorecard ensembles. But unlike some manifestations of AI, scorecards offer transparency and explainability. This is a big theme in any conversation about AI and credit risk––you need to be sure you understand how that decision was made. Most AI technology remains “black box” and can’t provide an answer when a customer asks, “How did I get this score?”
That said, if I can use machine learning to expose powerful and predictive new latent features of credit risk, I can then directly incorporate them into a scorecard model. This preserves transparency while improving prediction.
If you’d like to learn more about FICO scoring concepts, this webinar explores the FICO® Score and alternative data. And check out my Twitter feed, which is always rolling with my latest thoughts on analytics and AI.