9 Documentation Elements of a Model Risk Program
Models are all around us — integral and important to operational efficiency — but the risks that they sometimes pose can materially impact the financial well-being of even the most well-structured organizations. In order to understand the risks, we must first define what a model is and what the inherent risks are when operating a model.
A model refers to a quantitative method, system or approach that applies statistical, economic, financial or mathematical techniques and assumptions to process data into quantitative estimates. In other words, it’s a methodical way to process and sort data. A model consists of three components:
- An input
- A processing element
- An output
To build on that idea, model risk is the potential for the misuse of models to adversely impact an organization. Model risk primarily occurs for three reasons: a) data, operational or implementation errors; b) prediction errors; and c) incorrect or inappropriate usage of model results.
Models exist and assist in the identification, assessment and evaluation measurement. They can also assist in monitoring nonfinancial risks like behavioral analysis and financial risk like credit risk.
It therefore makes sense to warehouse these different models under a central Model Risk program. Such a holistic program can offer a consistent approach in addressing the challenges outlined below.
Challenges of a model risk program
Model risk programs can have the following inherent challenges:
- A lack of appreciation and understanding of the total number of models (inventory) being utilized by the organization
- Lack of a valid method to update the model inventory on a regular basis
- Inability to maintain the independence of the model validation department
- A model validation process that fails to demonstrate effective challenge, review and independence
- Lack of suitable data used in the model development process to support review/validation
- Lack of developmental evidence to substantiate model assumptions
- Lack of an explanation to support the application of expert judgment and model overrides
- Lack of validation or failure to re-evaluate at regular intervals
- Failure to maintain comprehensive and up-to-date model documentation
These challenges can be easily overcome by putting a Model Risk Program in place; such a program build out and size is determined by how material and critical the models are in influencing the business and strategic direction of the firm. The Model Risk program can be tooled and modulated to suit the specific needs of the firm. The key ingredient is the need to document, which will allow and afford quality testing and assurance around the key areas of validation and change management. The CFO should insist on a minimum standard with regard to model documentation, which will give both the CFO and the Board some reassurance and comfort around the deployment and utilization of models. Moreover, such documentation will be evidenced as one of the control factors in corporate governance. The documentation should provide a consistent set of standards, which articulate guiding principles that cover the model process and provide comprehensive guidance for practice and standards.
What to include in your model risk program framework
In order to address these challenges, I would recommend incorporating the following nine model risk documentation components that enable the framing of your model risk program.
1. A clear and consistent organizational narrative documented in writing covering principles, objectives, scope, model risk program design (including standards) model risk appetite, model risk taxonomy, controls and industry regulations.
2. A policy describing the oversight and governance which will include the roles and responsibilities of all stakeholders and participants.
3. Well-thought-out and actionable model risk management policies and procedures to include:
- Data management policy
- Model validation policy and requirements
- Model documentation requirements for both in-house developed models and third-party vendor models.
4. Policy and procedures to include the completeness of the current model inventory and process of updating on an ongoing basis.
5. Policy and procedures which will risk rate the model’s materiality to the function of the organization.
6. A model development and implementation policy which incorporates the following considerations:
- Integration into new products
- Planning for model updates and changes
- Planning for additional uses of existing models
7. A model validation policy which incorporates the following considerations:
- Evaluation of conceptual soundness, methodology, parameter estimation, expert and other qualitative data
- Assessment of data inputs and quality
- Validation of model outcomes
- Assessment of ongoing monitoring metrics and performance
- Model risk scoring
8. A documented model issue management and escalation process which will describe the issues, cataloguing of issues, issue remediation and action plans.
9. Disaster and contingency planning policy for approved models describing the Plan B in case of model failure, corruption or cyberattack.
For some tips on financial modeling standards see this piece, FAST Modelling – What’s it all about and why you should adopt it.
Passing thought.
As both technology and the use of models increases, there is a need to extend the principles of corporate governance evidenced by such documentation to mitigate model risk.
Identify your path to CFO success by taking our CFO Readiness Assessmentᵀᴹ.
Become a Member today and get 30% off on-demand courses and tools!
For the most up to date and relevant accounting, finance, treasury and leadership headlines all in one place subscribe to The Balanced Digest.
Follow us on Linkedin!
