All posts by Neill Crossley

Risk & Compliance Garbage In, Garbage Out: Correcting Sample Bias

Feb112015

On this blog, I’ve previously warned about the dangers of ignoring data sample bias, a problem often leading to predictive models that don’t perform to expectations. It’s an analytic example of “garbage in, garbage out” that, for lenders, can result in erroneous and often very expensive decisions, such as higher default rates due to extending credit to the wrong people. Today, I’ll focus a bit more on sample bias, referencing a case study to illustrate the risks. Specifically, I’ll share why using PD (probability of default) models developed strictly on accepted and opened applicants, while ignoring rejected or not booked applicants, can lead to subpar acquisition risk models. I pulled the case study from a new FICO white paper “How to Correct Sample Bias,” written by my colleague Nina Shikaloff. The case study is based on a loan origination scoring process that uses credit bureau data. This allowed us to... [Read More]

Leave a comment

Risk & Compliance Can Focusing on Basel II Increase Your Bank’s Bad Debt? Part 3

Compliance
Nov242014

I’ve been highlighting questionable modelling practices by some retail banks under the auspices of Basel II compliance, as well as offering alternate best-practice guidance. Today, in my third entry in the series, I’ll focus on the misuse of the Gini coefficient measure, which generally falls into two categories: “Chasing the Gini” Basel II has increased the focus on building more powerful predictive models. However, we often see this lead to an inappropriate focus on the Gini coefficient measure. At its worse, this may entail model developers blindly trying to maximise this measure of predictive power without understanding the consequence. Working with one client, for example, we were advised that they had increased the Gini by over 20 points on a new model. However, when the model was implemented, there was no improvement seen in business performance. As it turns out, the increase stemmed from impacts very high up the score... [Read More]

Leave a comment

Risk & Compliance Can Focusing on Basel II Increase Your Bank’s Bad Debt? Part 2

Compliance
Nov032014

I’ve been blogging about problematic modelling practices that may be creeping into some retail banks under the auspices of Basel II compliance. This time, I’ll focus on two areas related to sample bias, a problem which often leads to models that don’t perform to expectations: Not building models on the population they are going to be used on In the context of Basel II models, this usually manifests itself as not undertaking reject inference on origination-focused PD (probability of default) models. We hear phrases from clients such as: “We don’t do reject inference because Basel II only allows us to build on known outcomes.” Reject inference is the process of estimating how people whose credit applications were rejected would have performed had they been accepted. When building an origination PD model, if you only use applications that were opened, your development sample will be inherently biased (unless the existing accept... [Read More]

Leave a comment

Risk & Compliance Can Focusing on Basel II Increase Your Bank’s Bad Debt?

Compliance
Oct212014

Are bad modelling practices creeping into some retail banks under the auspices of Basel II compliance? I built my first credit scorecard nearly 20 years ago, and these days I regularly work with retail banks on their approach to the predictive model lifecycle, from undertaking health checks to improving efficiencies and insight. Recently for a few banks, a concern struck me: their interpretation of Basel II regulations seems to influence their credit scorecard development methodology in ways that could increase their bad debt levels. There are a number of aspects to this, but in this blog, I’ll focus on: The use of Basel II defined PD models as the primary credit risk assessment The primary purpose of Basel II PD (probability of default) models is to input into IRB Regulatory Capital Calculations, which in turn provide for comparative measures across organisations. Consequently, a standard default performance definition is required (90... [Read More]

2 Comments