Using Augmented Intelligence and Machine Learning in Credit Scoring

“For models for high-stakes environments, model performance is not all you need.” 
- Agus Sudjianto 
 

“Use machine learning by all means but as a copilot, don’t use it on autopilot.” 
- Gerald Fahner

At FICO World I took a deep dive with Agus Sudjianto, Wells Fargo’s Head of Corporate Model Risk, on critical objectives, proven methods, and advanced technologies and techniques for designing high-stakes credit scoring models responsibly. We were particularly focused on the use of machine learning models and artificial intelligence to assess credit risk. I’m indebted to Agus for a passionate discussion illuminating the credit scoring design process beyond what normally hits the artificial intelligence and machine learning headlines.

To Agus’ memorable mantra, “Model performance is not all you need.” I add, “Machine learning is not all you need”. We already know that machine learning and AI won’t make up for dataset limitations in credit scoring. But what we’re contemplating here is that (besides good data) it takes an effective combination of domain expertise and machine learning — the augmented intelligence approach to score development — to develop great credit scoring systems that work for borrowers and lenders.

FICO lists eight critical objectives for credit score development:

• Predictive
• Robust
• Transparent
• Credible
• Fair
• Private
• Compliant
• Rapid development

Reaching perfection simultaneously on all these credit scoring objectives may remain a utopia, since there could be trade-offs. Augmented intelligence helps us to understand and navigate possible trade-offs and to find the responsible balance that gives us the best tools for managing credit risk.

What does the augmented intelligence approach to developing credit scoring systems look like? This is a highly informative, iterative feedback process that interweaves and synthesizes complementary strengths of human intelligence and AI. 

The process gives humans an accurate understanding of what the machine learning models learn from the dataset. It allows data scientists developing credit scoring systems to spot early on when certain objectives may not be met by a model, and in response to refine the machine learning algorithm. For example, we might apply local constraints to the credit scoring model to ensure that only credible and sound predictive relationships regarding credit risk are being captured, free of bias.   

I really like Agus’ emphasis on requiring inherently interpretable machine learning algorithms for high-stakes decisions. This gives the augmented intelligence process super-powers because humans can comprehend exactly how credit scores are computed, avoiding the AI black boxes and difficulties with approximating them. (FICO has its own interpretable machine learning neural networks, which our chief analytics officer, Dr. Scott Zoldi, discusses in the post How to Make "Black Box" Neural Networks Explainable.) 

“For designing quality machine learning models you need to be very deliberate.” 
- Agus Sudjianto

No wonder then that we found much common ground conversing on our experiences with a range of AI and machine learning technologies that can fit this bill in credit scoring. Another important takeaway from our discussion was to balance predictive power and interpretability in credit scoring; we don’t need to compromise on either to develop powerful credit scoring models that benefit borrowers and lenders alike. We can have our cake and eat it too!

“You can have interpretability at the same time as very high predictive power and robustness.” 
- Gerald Fahner

Learn More About AI and Credit Scoring

• Read How to Build Credit Risk Models Using AI and Machine Learning
• Read Can Machine Learning Build a Better FICO Score?
• Read Fighting Bias: How Interpretable Latent Features Remove Bias in Neural Networks
• Read our white paper on Machine Learning and FICO Scores
• Discover our posts on Responsible AI
• Explore FICO scoring solutions