Can Focusing on Basel II Increase Your Bank’s Bad Debt? Part 3

I've been highlighting questionable modelling practices by some retail banks under the auspices of Basel II compliance, as well as offering alternate best-practice guidance.

Today, in my third entry in the series, I’ll focus on the misuse of the Gini coefficient measure, which generally falls into two categories:

1. “Chasing the Gini”

Basel II has increased the focus on building more powerful predictive models. However, we often see this lead to an inappropriate focus on the Gini coefficient measure. At its worse, this may entail model developers blindly trying to maximise this measure of predictive power without understanding the consequence.

Working with one client, for example, we were advised that they had increased the Gini by over 20 points on a new model. However, when the model was implemented, there was no improvement seen in business performance. As it turns out, the increase stemmed from impacts very high up the score range on a minority of applications, resulting in little overall impact on the credit risk decisions made.

All measures have their pros and cons, and as part of my best-practice recommendation, be sure you consider:

• A range of predictive measures when developing a model, such as divergence, K-S and KS score point/percentile, R-squared, etc.
• How the model will perform when implemented, looking at trade-off curves, etc.
• That you’re not “over-fitting” the model to the development data. Instead, focus on developing robust models that will perform well over time.
• Other factors such as model interpretability and implementation costs.

2. Setting of minimum Gini performance thresholds

Whilst it is important to set benchmark parameters that highlight deterioration or unstable performance, these need to be set in the context of what is appropriate for that model. We have seen models not approved by a bank's internal review committees because it may have a Gini of 38%, when the arbitrary benchmark is set at 45%. However, the model in question was part of a segmented model scheme; in combination with the other segmented models, it provided a much better overall assessment of the population in question.

This highlights a lack of understanding by the internal governance team that such measures can be influenced by the model design. Education on such factors is crucial to the successful review and validation of models.

If you missed my earlier posts in this series, be sure to check out my first and second entries. All the banks I’ve been blogging about have Advanced IRB status and experienced credit risk managers. That suggests one overall takeaway from this blog series: while Basel II compliance and credit risk management complement each other, one cannot be substituted for the other. Getting them to work together leads to better credit risk assessment and increased profitability.

For additional modelling best practices, I once again encourage you to download these two FICO ebooks: High-Performance Predictive Analytics and Comply and Compete: Model Management Best Practices (registration required).