Risk & Compliance Model Risk 101: A Checklist for Model Developers

Aug032015

I’ve been blogging about the need to ensure checks and balances are in place across the entire model risk management and governance process, an approach often referred to as the “three lines of defense.” In this post, I’ll focus on the first line of defense – model developers and users – providing some insights for enhancing productivity and effectiveness.

While banks strive to improve the efficacy of their models and the efficiency of their model development processes, few have implemented effective processes to: a) identify and prioritize models for redevelopment, and b) ensure that all modeling teams are following a standard process. As a result, while model quality at the individual or department level may be generally solid, areas such as documentation, variable usage, data inputs and assumptions, approvals, and other processes will likely vary, sometimes substantially, thereby introducing the potential for both adverse regulatory and business risk.

In addition, model developers often are pulled away from development efforts to perform tasks such as running validation reports and participating in regulatory/compliance efforts. In some organizations, business analysts who could be developing their own models are limited by the lack of user-friendly development tools, as well as not having access to the developers who are stuck in a compliance quagmire.

How do we develop (and deploy) models faster – and smarter?

With an ever-growing volume of data upon which to build models, a wider choice of predictive (and prescriptive) analytic tools, and demands from the business to apply analytics to more types of decisions, banks are poised to take the next leap in model development. Automation can free developers from time-consuming compliance activities, and workflow management systems can ensure that approved model development processes are being followed, while automatically capturing what regulators demand, in the form of consistency, supporting documentation and approvals/sign-offs for key project milestones.

An integrated solution melding analytics and technology should allow developers (and users) to:

√ Use their preferred modeling tools to rapidly and intuitively develop new predictive models based on business specifications – even for non-experts

√ Maintain a centralized repository for all models, providing quick access to model performance results, development details, key documentation and an audit trail of decisions made throughout the model lifecycle

√ Access a dashboard with at-a-glance status of all models

√ Create and run business process workflows to capture required documentation, and automate the document-intensive, approval-based model management process across departments and systems

√ Drive greater transparency for regulators by automating methodologies such as documenting data sample selection, segmentation, responses to economic events, characteristics, etc.

With the front line following a consistent process for developing models, you not only benefit from improved predictive models; you also enable the second line of defense – primarily model risk managers – to more readily identify and measure model risk, including validation testing, model usage, risk assessments and ongoing risk monitoring.

I’ll cover more about the work of this second line of defense in my next blog post. Until then, be sure to check out our Insights white paper Reducing Regulatory Drag on Analytics Teams, or visit http://www.fico.com/en/analytics/model-management-and-compliance to learn how other organizations are driving stronger model management with advanced analytics and automation.

1 Comment