How to Build Credit Risk Models Using AI and Machine Learning

Which works better for modeling credit risk: traditional scorecards or artificial intelligence and machine learning?

Given the excitement around AI today, this question is inevitable. It’s also a bit silly. While some new market entrants in the fintech industry may have a vested interest in pushing AI solutions, the fact is that traditional scorecard methods and AI bring different advantages to credit risk modeling — if you know how to use them together.

Take, for example, our well-established scorecard technology leveraging our AI and machine learning to build better credit risk models, algorithms that analyze the probability that customers will pay on time.

How FICO Uses AI to Build Better Credit Risk Models

When FICO develops credit risk models for, say, credit originations, these models are segmented — different types of customers and different credit products require different models to assess their credit risk.

In traditional risk modeling, customer segmentation is based on “hard” lines and broad factors, such as new customer vs. existing customer. This doesn’t capture the behavior of certain individual entities or more optimal ways to segment scoring models.

To build models in a segmented solution, our data scientists often use AI and machine learning algorithms in their process to discover a better way to segment the scorecards. This allows us to apply AI to improve risk prediction without creating “black box” models that don’t give risk managers, customers and regulators the required insights into why individuals score the way they do.

Our data science teams are now using techniques such as collaborative profiles to reveal entity segmentation based on customer behaviors. We can then group customers into micro-segments based on that similarity, instead of typical segmentation approaches that rely on hard business attributes. For example, collaborative profiles derive behavioral archetype distributions — these could include archetypes that point to credit seekers building credit histories vs. those who have higher risk and covering misuse of credit elsewhere in their history.

The way that we can capture these subtle changes in behavior, and can incorporate them into the credit risk model, presents a distinct advantage for FICO customers. Our approach builds on mature, time-tested analytic models and scorecards, enhancing them with advanced AI technology to drive better segments and feature creation in models.

Another approach is to use AI and machine learning to “train” models to discover maximum predictive power, and find new relationships amongst input features that could produce a stronger model. For example, consumer credit utilization is always an important feature in a credit model, as is delinquency, but a nonlinear combination of these can produce more optimal results in a machine learning model. You can then drive these new inputs into a traditional scorecard model to ensure explainability in ways that regulators are accustomed to.

However, predictive power must not come at the expense of explainability. Some people will say, “The bigger machine learning model has to be better. It learns more things about the data.” More complicated models that are not interpretable, particularly in production, generally don’t perform any better. They’ve learned noise about the modeling data set and biases, and generally will not deliver robust performance out of time when the model goes into production. Thus, a key part of using AI responsibly is not only building a scorecard or machine learning model that is interpretable and ethical, but also ensuring that it continues to be responsible in the production environment.

Improving Risk Management Results with AI and Machine Learning

The two examples below illustrate how you can achieve better performance by combining machine learning and scorecard approaches to improve traditional scorecards.

When developing a credit card churn model, FICO data scientists used machine learning to discover a powerful interaction between recency and frequency of card usage. The option to include this interaction as a nonlinear input feature in an interpretable fashion into a scorecard led to a substantial improvement (~10%) of the lift measure, used to characterize the performance of attrition models. In addition, another 15% performance improvement was found by applying machine learning with a much larger set of features relating to event-specific recencies and frequencies. These predictive improvements in turn can translate into substantial portfolio profit gains for a much more precisely targeted retention strategy.

In a project to build a home equity portfolio with limited data, the lack of enough “bads” (poor performing loans) in our dataset was causing some problems. By building a machine learning score with optimized hyperparameters, our data science team was able to confirm that we were losing a significant amount of signal with a traditional scorecard. Using machine learning led us to change the model performance outcome from a binary outcome to a continuous outcome. By combining this technology with scorecard technology, we created a strong, robust, palatable solution and saw a 20% improvement in model performance (KS) over a traditional scorecard model alone (see below).

Use AI as a Tool

FICO has long been involved with using AI and machine learning as part of our analytics approach. How long? We recently filed a new explainable AI patent application to improve upon the IP of a FICO explainable AI patent granted in 1998 — that patent already expired.

Years of market experience have validated our approach, which is very different than the “move fast and break things” brashness we see from some AI start-ups in the credit space. They’re focused on using many types of alternative data, such as information gleaned from social media, to deduce credit risk.

Innovation is great, but you don’t want to naively throw in a lot of new data sources — many of which may not be permissible in credit decision-making, and might be easily manipulated (like social media data) — into any old AI model that comes up with a score that may not be explainable or impute bias. Why not? First of all, lenders in many markets do need to be able to explain how a customer was scored. Second, if you don’t really understand what relationships are being learned from this data then you shouldn’t be using the model.

For those credit professionals who are hesitant to move from scorecards to using machine learning, fear not. For more than five years, FICO has steadily delivered research-based innovation in the area of Responsible AI — specifically, purpose-built machine learning algorithms for credit risk that mirror the benefits of scorecard technology. These interpretable neural networks are palatable, interpretable, ethical and auditable.

FICO’s Responsible AI work demonstrates the ability to expose and eliminate biases derived from data showing up in the model. Even in cases where the data has been purposely skewed toward one subset of the population over another in the research lab, the method minimizes pick-up of those biased signals versus core relationships that matter. Our patented technique removes data noise and irrelevant correlation, where bias can be expressed.

Although it’s ill-advised to blindly build any old machine learning model in an area as impactful as credit risk, interpretable machine learning models can consume varied complex data sets more rapidly and responsibly than traditional ML models. FICO’s interpretability method can automatically surface and triage biases expressed in a machine learning model; this helps ensure the operationalization of human-in-the-loop development practices rooted in palatability, explainability and ethics, rather than leaving these important requirements to sheer luck or, worse yet, leaving them out altogether.  

FICO’s ongoing work in Responsible AI is explored thoroughly in my series of LinkedIn Live talks with other experts in the global AI community.

The Power of Scorecards

Scorecards are a powerful tool because, like AI, you can incorporate non-linearity in the input layer, and you can take advantage of different features that may be predictive in different ways for different subpopulations, by using segmented scorecard ensembles. But unlike many manifestations of unconstrained AI, scorecards offer transparency and interpretability. This is a big theme in any conversation about AI and credit risk — you need to be sure you understand how that decision was made in order to lend responsibly. Most AI technology remains “black box” and can’t provide an answer when a customer asks, “How did I get this score?”

That said, if I can use machine learning to expose powerful and predictive new latent features of credit risk, I can then directly incorporate them into a scorecard model which customers are accustomed to using (and their regulators are accustomed to reviewing). This preserves transparency while improving prediction, and ultimately risk management.

For more information on trends in AI and machine learning, check out my LinkedIn page and Twitter account, which are always rolling with my latest thoughts on analytics and AI.

How FICO Can Help You Advance Your Credit Risk Models and AI

• Watch my LinkedIn Live conversations on Responsible AI
• Discover FICO’s AI, machine learning and advanced analytics solutions
• Read FICO’s posts on Responsible AI
• Download our AI Playbook: A Step-by-Step Guide for Achieving Responsible AI
• Explore FICO Platform

This is an update of a post first published in April 2017.