Welcome to the latest Model Management Monday. This is the fourth in my blog series on model management, each post highlighting a best practice that supports both compliance and improved performance.
Best Practice #4: Choose the Right Model Type
Financial institutions should select a model type appropriate for data type and decision area, and one that will provide robust predictions. For both business and regulatory purposes, you should also consider the following:
- Transparency. Your model type should be easy to understand and explain, both internally as well as externally to regulators and customers. Look for interpretable features that allow you to identify and explain what is driving a score result. A risk model should include reason codes, which many regulators require you give to customers when declining a request for credit. Reason codes identify the factors that had the greatest negative impact on the score.
- Palatability. Regulators will ask about model outcomes, so it is important that model scores and reason codes have a high degree of face validity. Palatability is about intuition and common sense, not complex mathematics. Does your model behave intuitively from a business context and is it directionally correct? For instance, as length of good credit history increases, does the risk score improve?
- Ease of engineering. During development, you may need to engineer or fine-tune a model to ensure it will address your identified business goal. This may require you to substitute or remove predictive characteristics that, while predictive, may cause regulators or customers to raise objections. You may wish to alter variable binnings or apply pattern constraints to mitigate the impact outlier values, smooth noisy data, and improve the model's robustness.
Bottom line, you’ll want to select a model type that’s sensible to regulators and customers, and can be easily re-engineered when needed. As an example, scorecards are commonly used model types in risk modeling because they are highly transparent and interpretable. The sample selection of a scorecard above shows how points are assigned for different values within a category of information. There’s a clear connection between each factor and how it correlates to an individual's score. This not only enables the model to generate useful reason codes for consumers, but the information can be used to describe the model to regulators or answer internal questions.
For more details on this and other best practices, download the FICO Insights white paper, "Comply and Compete: Model Management Best Practices" or Martin Butler’s paper on Model Management and Governance. And check our blog next Monday for my next post in this series.