Evolving Landscape of Machine Learning Relative to the FICO® Score
How does progress in artificial intelligence stack up to the FICO credit score model?
Artificial intelligence (AI) was around long before the start of my career, and back in the ‘90s, analyst tools and techniques were rooted in AI paradigms invented decades earlier. Early on during the evolution of AI, credit risk scores started out as small and manually designed linear discriminant models. In later years, when I turned toward applied business analytics, data and models grew larger. Nonlinear models, sometimes augmented with automatically discovered complex predictive features, became more widespread.
Before I delve into the evolving landscape of machine learning relative to the FICO® Score, it is crucial to clarify that AI is not used in the final version of the FICO® Score algorithm. However, AI is employed during the research phase to help identify data that has the potential to be highly predictive. This approach contributes to improving the overall effectiveness of the FICO credit score model.
Recently, I caught up with an industry leader, Rob Jasper, Advisor for the Pacific Northwest National Laboratory, for a thought-provoking discussion about the historical impact of AI and its influence on credit scoring. We drew comparisons between FICO’s augmented intelligence philosophy and the common perspectives on the use of AI and machine learning in applied business analytics. Rob took stock of the astonishing advancements of AI and machine learning spanning more than six decades. He explained that the latest advances have increasingly focused on sheer predictive power, often to the detriment of explainability, and given way to the black box problem. I expanded on Rob's insights by contextualizing how the human-centric modeling culture that underpins the FICO® Score has evolved for several decades to the need for providing credible explanations, not just predictions, foreshadowing some more recent developments in the field of explainable AI/machine learning.
After defining AI as “computer systems capable of performing complex tasks that historically only a human could do,” Rob explained that this definition is a moving target because technologies that were regarded as AI decades ago are now regarded as computer science. Intellectually demanding tasks once viewed as only achievable by humans were shockingly accomplished in a superior manner by AI and then subsequently became normalized as computer science, such as handwritten digit recognition or playing chess, leaving the ever-shifting forefront of AI to keep tackling more complex tasks.
Developing a new credit risk scoring algorithm for the FICO® Score is an example of a complex modeling task that we accomplish with a diverse team of experienced data scientists and subject matter experts. Ironically, what makes the task so complex and dependent on domain expertise is that the credit scoring algorithm must be transparent, reliable, and intuitive for all human stakeholders in the financial ecosystem, including the borrower, lender, regulator, and investor.
The radar chart below lists multiple objectives for the FICO scoring algorithm: It needs to be highly predictive and this score power needs to be robust in the face of data drift as economic tides are ever shifting and the consumer credit landscape is always evolving.
The scoring algorithm also needs to be explainable and credible for humans and should be expeditiously developed and deployed. Fairness, privacy, and compliance are non-negotiable requirements. Tradeoffs may exist between certain objectives where they may oppose each other. Striking the responsible balance is a fundamental requirement for the FICO® Score.
Professor Leo Breiman, a statistician and machine learning pioneer, highlighted important tradeoffs between model development objectives in his landmark paper “Statistical Modeling: The Two Cultures.” For example, Breiman’s random forests, a class of machine learning models, might be expediently trained in the algorithmic modeling culture. This approach, with minimal human involvement in the model training phase, aims to achieve greater accuracy and predictive performance than a regression model painstakingly developed by a statistician adhering to the data modeling culture while focusing on objectives of inference and explanation, rather than prediction. The machine learning model might then be harder to explain than the regression model, as the red and blue lines in the radar chart illustrate. Rob agreed, vividly describing differences between members of the two modeling cultures in their attitudes towards modeling.
The objectives on the radar chart are a simplification. Underneath some objectives, there may be further sub-objectives. For example, the FICO® Score must be highly predictive for numerous use cases characterized by alternative prediction targets (e.g., default on a car loan or credit card payment), similar to the sub-field of multi-task learning in machine learning.
To satisfy the requirements for the FICO® Score and to strike the best balance across multiple objectives, FICO’s data scientists leverage inherently interpretable and constrainable, multi-objective machine learning techniques for credit risk model development to capture robust predictive signals from the data sources in the form of explainable predictive features that drive the FICO® Score. Data scientists and domain experts are in the driver’s seat to steer machine learning toward a responsible balance. Enhancing the perspective of Breiman’s paper, we embrace a “third culture” of augmented intelligence rather than artificial intelligence, which sits between the two extreme cultures highlighted. The augmented intelligence approach tailors the scoring algorithm to human needs and is done with the help of machine learning to add effectiveness and efficiency to the tedious parts of model development. Developing a new FICO® Score is not something today’s mostly data-driven artificial intelligence could accomplish without major human guidance, which leads to my claim that “AI alone just doesn’t cut it.”
We are continually advancing our algorithms, tools, and processes to improve on all objectives on FICO® Scores. A recent example targeting the “expeditious” objective is “data sculpting.” This novel capability we’re piloting in our FICO® Scores Innovation Lab initiative shows promise to help accelerate lender adoption of new credit scoring models. Lender-specific applicant populations often differ from FICO's development baselines. Data sculpting adjusts FICO's development population to match the distribution profiles of individual lenders, resulting in more accurate simulation outcomes and improved KPI projections – allowing lenders to reduce time to value for a new or updated credit scoring model.
Learn More About Augmented Intelligence and Data Sculpting:
Watch my full discussion with Rob Jasper here: The Evolving Landscape of Machine Learning Relative to the FICO® Score 2024 | FICO
View the presentation deck here: The Evolving Landscape of Machine Learning Relative to the FICO® Score Presentation
Popular Posts
Average U.S. FICO® Score at 716, Indicating Improvement in Consumer Credit Behaviors Despite Pandemic
The FICO Score is a broad-based, independent standard measure of credit risk
Read more
The GSE Data Is Public. Here's What It Shows About Credit Score Modernization.
Independent Milliman analysis of GSE historical data confirms what anyone can now verify directly: FICO® Score 10T is the most predictive score.
Read more
FICO® Score High Achievers: Is Age the Only Factor?
<p>FICO’s latest research on <a href="http://www.fico.com/en/blogs/risk-compliance/us-credit-quality-rising-the-beat-goes-on/" target="_blank">national FICO® Score distribution</a> shows that US credit quality continues to trend upwards, with an increasing number of consumers scoring in the highest scores ranges. Given that, we decided it was time to refresh our study on “<a href="http://www.fico.com/en/blogs/risk-compliance/infographic-what-does-a-high-credit-scorer-look-like/" target="_blank">FICO® Score high achievers</a>,” where we examine the credit behavioural profiles of consumers with higher credit scores.</p> <p>As expected, our latest study showed that older people generally have higher scores, as has been the case in our <a href="http://www.fico.com/en/blogs/risk-compliance/age-before-beauty-credit-worthiness-before-youth/" target="_blank">past research</a>. This is largely because they have trade lines that have been open longer, which leads to higher scores (assuming the trade lines are in good standing and all other things equal). The most credit savvy among you will remember that length of credit history accounts for roughly 15% of the overall <a href="http://www.myfico.com/CreditEducation/WhatsInYourScore.aspx" target="_blank">FICO® Score calculation</a>.</p> <p>But for younger consumers, it isn’t too helpful to say, “just wait until you are 50 years old, and that will fix things!” Thus, I decided to dive deeper with our analysis, including looking at the credit behavioural characteristics of younger consumers with high FICO® Scores.</p> <p>In this analysis, I looked at credit characteristics for the two highest score bands, 750-799 and 800-850, which have about 16% and 23% of the population, respectively. I further broke this down by looking at the characteristics of the youngest 15% in each group, which occurs at age 29 for the 750-799 group and 43 for the 800-850. We used FICO<strong>®</strong> Score 9 for our analysis, and examined typical variables of the categories that go into the make-up of the score.</p> <p><a href="http://www.fico.com/en/blogs/wp-content/uploads/2016/10/FICO-Score-High-Achiever-table.png"><img class="alignnone size-full wp-image-29367" src="http://www.fico.com/en/blogs/wp-content/uploads/2016/10/FICO-Score-High-Achiever-table.png" alt="fico-score-high-achiever-table" width="715" height="747" /></a></p> <p>Since there’s a lot to dissect in this chart, I’ll highlight a few findings that struck me: <ul> <li><strong>Credit history.</strong> Higher scores are undoubtedly associated with a longer time on file. What’s striking is the higher average time on file for the 800+ group, both overall and for revolving accounts (the most common trade line). Clearly, it helps to start credit use early in life – but that’s not the sole determining factor for high scoring. The younger cohorts in both the 800+ and 750-799 groups have time on file medians significantly less than their older counterparts.</li> <li><strong>New credit.</strong> High credit scorers do, in fact, pursue new credit. We see little variation in months since last trade line opening and last inquiry among all groups observed.</li> <li><strong>Pay on time.</strong> As you might expect, the further in the past the delinquency, the higher the score – and that’s <u>if</u> a delinquency ever happened. 95% of the 800+ people and 72% of the 750-799 group have never been delinquent, which is a striking contrast to the 57% for the total population. The percentages for the younger groups are even higher, especially so for the 750-799ers. Collection items are also scarce for these high scorers. While 23% of the national population have collection items greater than $99, the figures are only 0.5% for the 800+ group and 3% for the 750-799 group. It is clear that, in conjunction with starting a credit file early, paying on time is a key trait of the FICO® Score high achiever.</li> <li><strong>Balance and utilization.</strong> Finally, we look at how credit is being used, focusing on revolving product use. (Since most people only have one mortgage or one auto loan, variation in instalment product figures is minimal and driven by payment history, which is near perfect for high scorers, as we’ve seen.) FICO® Score high achievers tend to have more open revolving accounts. While the % active trades and utilization are generally lower, by no means do high scorers avoid the use of credit products. What drives the difference here is the amount of <u>available</u> revolving credit utilized. The 800+ group is roughly 1/4 of the national median, both in total and on their highest utilized card; they also have lower balances and more cards<em>. </em>The 750-799 group is more active and higher utilized, but still only at 2/3 of the national figures. The young 750-799 group compensate with lower revolving balances and, as a result, lower utilization, which is in line with what we see for the 800+ group.</li> <li><strong>Benchmarks.</strong> I’d like to point out that while this study provides good benchmarks of the typical credit traits of the FICO® Score high achiever, individual credit profiles will vary widely. And keep in mind that when we sort groups of consumers as a whole by different behaviours, some results may look superficially odd. For example, the 800+ band has a longer median average open account time (128 months) than average revolving months on file (140 months).</li> </ul> So, what do we conclude? A huge takeaway is that high scorers don’t have too much in the way of serious delinquencies and negative public record data.</p> <p>In most respects, the young 800+ people are like their peer group. They really only differ markedly in their shorter credit history and higher non-mortgage balance, both of which are a function of life stage. Our results show that you don’t have to be a credit product hermit; these high scorers have more credit cards. The takeaway is to start credit activity early and maintain well-managed and moderate usage.</p> <p>In many ways, the young 750-799 people are more interesting because their age profile is much younger. Their credit characteristics differ more from their peer group, particularly in terms of length of credit history. They compensate by having almost pristine credit records and lower balances.</p> <p>Ultimately, 39% of US consumers are FICO® Score high achievers, which they accomplish by following common-sense credit practices. It ain’t rocket science!</p>
Read moreTake the next step
Connect with FICO for answers to all your product and solution questions. Interested in becoming a business partner? Contact us to learn more. We look forward to hearing from you.