Welcome to our last Model Management Monday. This is the ninth and final part in my blog series on model management, each post highlighting a best practice that supports both compliance and improved performance.
Best Practice #9: Document Thoroughly
With the increased use of predictive models in automated decisioning, regulators worldwide place tremendous importance on documentation and oversight. To answer regulator questions for details of how a particular model was developed, along with who approved the model at key stages of its development, you need the right tools in place to quickly retrieve the supporting evidence.
With that in mind, you should keep an inventory of every model within your operating environment, cataloguing its purpose, usage and restrictions on use. Technical documentation should be captured describing the modeling methodology used, along with supporting evidence for why the final model inputs were chosen. Your documentation should be detailed enough so that anyone unfamiliar with the model can understand how it operates, its limitations and your key assumptions. You also should be able to retrieve documentation for any vendor-supplied models and demonstrate that you understand it.
Your inventory should indicate the current health of each model (e.g., performing as expected, on a watch list, flagged for redevelopment, retired). Prior validation results should be readily accessible, along with any supplemental analyses that were performed. The model should also capture names of key stakeholders, such as the model manager and business owner.
You should also have a complete audit trail of changes to the model, capturing who made the change, along with the purpose of the change. All annotations and justifications should be digitally captured and attributed to an individual, and the sequence of model changes should be apparent.
By following approved process and implementing technologies that automatically ensure documentation and validation results are captured and persisted correctly and consistently, your institution’s highly trained analysts can focus on activities with greater impact, such as new model developments, rather than being consumed with producing standard validation and tracking reports. And by centrally documenting a model's design and limitations, you reduce risk of misapplying a model.
For more details on this and other best practices, download the FICO Insights white paper, "Comply and Compete: Model Management Best Practices" or Martin Butler’s paper on Model Management and Governance.
