GUEST BLOGGER

John Thackeray
Founder and CEO of Risk Smart Inc.

Models are all around us — integral and important to operational efficiency — but the risks that they sometimes pose can materially impact the financial well-being of even the most well-structured organizations. In order to understand the risks, we must first define what a model is and what the inherent risks are when operating a model.

A model refers to a quantitative method, system or approach that applies statistical, economic, financial or mathematical techniques and assumptions to process data into quantitative estimates. In other words, it’s a methodical way to process and sort data. A model consists of three components:

An input
A processing element
An output

To build on that idea, model risk is the potential for the misuse of models to adversely impact an organization. Model risk primarily occurs for three reasons: a) data, operational or implementation errors; b) prediction errors; and c) incorrect or inappropriate usage of model results.

Models exist and assist in the identification, assessment and evaluation measurement. It can also assist in monitoring both nonfinancial risks like behavioral analysis and financial risk like credit risk.

It therefore makes sense to warehouse these different models under a central Model Risk program. Such a holistic program can offer a consistent approach in addressing the challenges outlined below.

Challenges of a model risk program

Model risk programs can have the following inherent challenges:

A lack of appreciation and understanding of the total number of models (inventory) being utilized by the organization
Lack of a valid method to update the model inventory on a regular basis
Inability to maintain the independence of the model validation department
A model validation process that fails to demonstrate effective challenge, review and independence
Lack of suitable data used in the model development process to support review/validation
Lack of developmental evidence to substantiate model assumptions
Lack of an explanation to support the application of expert judgment and model overrides
Lack of validation or failure to re-evaluate at regular intervals
Failure to maintain comprehensive and up-to-date model documentation

What to include in your model risk program framework

In order to address these challenges, I would recommend incorporating the following nine model risk documentation components that enable the framing of your model risk program.

9 risk documents that help frame the model risk program

A clear and consistent organizational narrative documented in writing covering principles, objectives, scope, model risk program design (including standards) model risk appetite, model risk taxonomy, controls and industry regulations.
A policy describing the oversight and governance which will include the roles and responsibilities of all stakeholders and participants.
Well-thought-out and actionable model risk management policies and procedures to include:
a.    Data management policy
b.    Model validation policy and requirements
c.    Model documentation requirements for both in-house developed models and third-party vendor models.
Policy and procedures to include the completeness of the current model inventory and process of updating on an ongoing basis.
Policy and procedures which will risk rate the model’s materiality to the function of the organization.
A model development and implementation policy which incorporates the following considerations:
a.    Integration into new products
b.    Planning for model updates and changes
c.    Planning for additional uses of existing models
A model validation policy which incorporates the following considerations:
a.    Evaluation of conceptual soundness, methodology, parameter estimation, expert and other qualitative data
b.    Assessment of data inputs and quality
c.    Validation of model outcomes
d.    Assessment of ongoing monitoring metrics and performance
e.    Model risk scoring
A documented model issue management and escalation process which will describe the issues, cataloguing of issues, issue remediation and action plans.
Disaster and contingency planning policy for approved models describing the Plan B in case of model failure, corruption or cyberattack.

The payoff of so much documentation

Model risk is very real, and I’ll be honest — it requires a heavy lift in documentation. But by covering your bases, the documentation will provide a consistent set of standards, which articulate guiding principles that cover the model process and provide comprehensive guidance for practice and standards on an enterprise-wide level.

John Thackeray is the founder and CEO of Risk Smart Inc., a consulting firm that specializes in the writing of risk documentation. Over his long career, he has held many risk positions, including CRO posts at Societe General and Penson Worldwide Holdings, where he interacted and engaged with U.S. and European regulators.

Featured