Survey: Major Gaps in Model Risk Management

We’ve been blogging about our recent survey, conducted with Chartis, where we queried global financial institutions (FIs) about their capital adequacy programs. As part of the survey, we asked FIs about their practices in model risk management. What we discovered is that following industry-standard best practices is the exception, not the norm.

Regulators mandate that all financial institutions have adequate processes in place for model risk oversight and control of every analytic model utilized throughout the credit lifecycle. Industry best practice is to develop such practices using a tri-layered defense:

1. Solid and effective controls exerted by the business, including formal processes for definition, development, implementation and ongoing monitoring of models.
2. Enterprise risk functions and committees that establish standards for model governance, validation and monitoring of adherence to established policies.
3. Independent audit and assessment of both the design and effectiveness of the controls and policies from the first two lines of defense.

Astonishingly, our survey highlighted that model risk governance is still seriously under-developed across FIs, with governance often not going past the first and second lines of defense.

Model risk management

Source: Chartis Research 2015

For instance, only 25% of tier 1 companies stated that responsibility for model risk governance has been given to the risk committee. Fortunately, both tier 3 and 4 companies appear to be more mature in their approach.

Additionally, less than 14% of all respondents, regardless of institutional size, involve their own internal audit teams in model risk issues. This highlights a significant and serious gap between business process and Basel guidelines, and thus is something that should be addressed by FIs rather urgently. Basel guidelines require internal audit functions to have responsibility for the approval and maintenance of risk models. This includes all aspects of verification of the consistency, timeliness, independence and reliability of the model, as well as the data sources used in these models.

One crucial part of effective model management is having full separation between the development and validation of models. These two elements cannot be conducted effectively within the same team or business unit. Fortunately, 96% of Tier 1 institutions surveyed have this separation in place. Contrast that with only 49% of Tier 3 institutions, which is of significant concern.

Also central to model risk management is the assurance of data quality. Surprisingly, less than 30% of survey respondents have a dedicated data governance program separate from IT. For most, data quality challenges are not addressed at the “C” level (i.e., ownership does not belong to CIOs, CROs or CFOs). This lack of central control/governance may lead to the lack of standardization of attributes, models and data quality issues at the group level.

How is your firm approaching data quality challenges?

Source: Chartis Research 2015

Stay tuned to our blog where I’ll continue to discuss key learnings from this survey. For full results, you can download the newly published report: Leading Practices in Capital Adequacy.

Thanks to my colleague Joanne Gaskin for co-authoring this post with me.