In my last post, I introduced the three critical analytic questions that must be answered before scoring more consumers, and shared recent FICO research around the first question. In this post, I'll discuss research addressing the second question: Is there sufficient credit repayment history to predict future repayment behavior?
Addressing this question requires a quick review of Predictive Modeling 101. At its core, a credit score evaluates past behavior to predict future behavior. To build a model that predicts future repayment behavior, you need to observe some repayment behavior—otherwise, what are you modeling?
To develop the FICO® Score, we look at credit behavior at two different points in time, 24 months apart, as shown in the graphic.
The consumer credit file at Snapshot A is used to calculate the score’s predictive variables. The same consumer’s credit file at Snapshot B is used to categorize the consumer as a good payer or a bad payer based on how he (or she) paid his credit obligations during the 24-month window.
Only consumers with actual credit repayment history between Snapshots A and B—what modelers call “classifiable performance”—can be included in the development sample. Makes sense, right? If the purpose of a model is to predict good payers and bad payers, then you would need to observe enough good payers and bad payers on the development data in order to build a meaningful model.
What does this have to do with scoring more consumers? Well, many currently unscorable consumers don’t have classifiable performance during the defined time period.
Take, for example, consumers with one or more collections or adverse public records on file at Snapshot A, but no prior credit account history. Our research revealed that only 18.8% of these consumers had classifiable performance during the subsequent 24 months. That means 81.2%, or four out of five records, would be excluded from the model development process.
Generating a score for the 81.2% based on the 18.8% with classifiable performance amounts to assuming that the risk patterns of a minority hold true for the larger group. Probably not a reasonable assumption.
We know that at the time of application (Snapshot A), consumers in this segment would not have had a FICO® Score, and the only information on their credit file would be negative (i.e., collections and/or adverse public records). To be approved for credit, they probably had to meet a lender’s “no FICO Score” underwriting criteria. This would likely have involved verifying substantial income or assets, and/or being offered credit products with tight risk controls such as secured cards.
Consequently, it seems unlikely that the repayment behavior of these 18.8% “cherry picked” consumers would align closely with the behavior and risk of the other 81.2%. A credit score built from such a sample is at risk for significantly overstating the credit quality of these newly scorable consumers. In the long term, this could harm consumers since they might be afforded credit that they couldn’t responsibly manage, and it could harm lenders since they might see higher losses than expected.
I encourage you to download our just-released Insights white paper (registration required) with our latest research on the issue of scoring more consumers. And next time you hear that scoring more consumers is unequivocally better, pause and ask if it was done prudently. At the end of the day, what’s most important for consumers and lenders alike is not the fact that a score can be produced, but that a meaningful score can be produced. To achieve the latter, it's essential to maintain rigorous modeling practices.
