“Supervisory Guidance on Model Risk Management,” issued by the Office of the Comptroller of the Currency (OCC) and Federal Reserve in April 2011, raised the bar for banks’ model risk management (MRM) programs. Banks that are implementing an MRM framework must overcome many challenges, including:

• Gaining organizational buy-in.
• Adopting tools and templates.
• Developing metrics for determining whether a model is working as intended.
• Facilitating risk-based model validations.
• Recruiting staff and resources.

In the five years since the guidance was issued, we have found that most banks struggle with implementation in three areas:

1. What’s a model? The guidance defines a “model” as a “quantitative method, system or approach that applies statistical, economic, financial or mathematical theories, techniques and assumptions to process input data into quantitative estimates.” (To read the guidance see OCC Bulletin 2011-12 or Federal Reserve Supervision and Regulation Letter 11-7.) This seems straightforward, but determining what should be considered a model is tricky. A model should be clearly identified and defined in an organization’s policy to make sure the model inventory is complete and accurate. Models have specific features: They use theories and assumptions, and the results are estimates. Also, models are used repetitively in support of important business decisions.

2. Validation or review? All models should undergo a full validation process at some fixed interval. The intensity and frequency of validation work should be based on the model’s risk, but validations should assess the soundness of all model components—inputs, processing and outputs. Validation staff should possess the requisite technical competencies and sufficient stature within the organization to effectively challenge the model developer and operate independently from model development and use to preserve objectivity.

The guidance also directs banks to “conduct a periodic review—at least annually but more frequently if warranted—of each model to determine whether it is working as intended and if the existing validation activities are sufficient.” The annual review for each model should affirm previous validation work, suggest updates to previous validation activities or call for additional validation activities. The annual review also should include an assessment of model performance results, of the logs where changes made to the model are recorded, and of business environment changes. Material changes may trigger limited testing or partial revalidation.

3. Model documentation. According to the guidance, “[d]ocumentation and tracking of activities surrounding model development, implementation, use and validation are needed to provide a record that makes compliance with policy transparent.” As such, banks should use a standardized approach for documentation across all models, although documentation for vendor models could be slightly different than in-house model documentation. Documentation templates are real time-savers. Also, a centralized repository provides an enterprisewide view of MRM policy, program and guidelines documentation. Banks need to collect, retain and update model documentation regularly. Version control and the ability to search documentation electronically are two critical document management must-haves.

Also, consider the purpose of the document and the target audience when developing documentation because some users are more technically proficient than others. Overly technical or complex topics might be more appropriate for appendices or supporting documents than for overview procedures. Documentation frequently is “low-hanging fruit” when it comes to audits, validations and regulatory examinations. Make sure your institution has a process to test its documentation for policy compliance.

The Role of Technology
Examiners expect banks to have a complete understanding of model risk throughout the enterprise, to confirm that validations are completed and issues remediated, and to employ a consistent approach throughout the enterprise. It’s undeniable that technological tools can help manage the depth and breadth of MRM activities.

Technology can efficiently provide essential support for key MRM processes, especially for banks facing human resource constraints. A bank can achieve a centralized and consistent approach by using technology to maintain and manage model information, model risk policies and model framework information. Technology also reinforces compliance with bank policies and procedures and simplifies reporting.

Don’t rush into the first technology solution that comes along, though. Take the time to develop a set of detailed requirements for both current and future needs. Determine the information and intelligence required from the system, including reporting, analytics and dashboard requirements. Identify the processes and procedures that can be automated or improved with technology, and try to automate those manual processes that are time consuming and most critical.

Go Beyond Validation
MRM practices should extend beyond just performing validation. Don’t forget governance, effective challenge and performance monitoring. Plot your course and establish more efficient MRM processes that align with today’s regulatory expectations.