Stress Testing: Top Down or Bottoms Up?

Stress_Test_1-9-13.jpgRecent rule making for expansion of stress testing requirements to new constituencies and Basel III implementation have stirred uncertainty and angst among banks. Stress testing regulation includes significant detail on reporting templates, model management and capital planning guidance. Standards of practice are beginning to form.  However, regulatory guidance provided on methodological and modeling approaches to date has created confusion and caused a divergence of practice and a variety of modeling approaches among financial institutions. The appropriateness of each approach is widely debated. 

One point often lost in the discussion is that multiple approaches are suggested by the regulators’ Interagency Guidance on Stress Testing (SR 12-7) published in May, 2012, to properly manage and control model risk:

“An effective stress testing framework employs multiple conceptually sound stress testing activities and approaches.”

This guidance applies across the capital planning process, including credit loss estimation, liabilities, new business volumes, and pro-forma balance sheet and income statements. In this article, we limit our discussion to two conceptual approaches for modeling stressed credit losses: top-down and bottoms-up. 

In top-down modeling, exposures are treated as pools with homogeneous characteristics. Scenarios (i.e., macroeconomic or idiosyncratic, event-driven) are correlated to historical portfolio experience. The outputs from this approach are intuitive and easily understood outside of the credit risk function and can be readily calibrated and back-tested against on-going actual and projected performance. This is increasingly important as stress testing and capital planning requirements are forcing stress-testing analytics to be coordinated among treasury, finance and risk groups.

Top-down approaches can also be easier to develop since pool modeling is not exposed to idiosyncrasies or noise of modeling single-firm financial statements.  Additionally, historical data is readily available at most institutions since the same type of data is needed for modeling allowances for loan losses. A bank’s own loss experience can, therefore, be incorporated into the analysis, satisfying an element of the “use-test” criteria to validate models, which is so critical for management and regulators. 

Top-down modeling is well adopted for retail portfolios where homogeneous groupings are more easily identifiable. This approach can ignore important risk contributors and nuances for more heterogeneous portfolios (e.g., commercial real estate, commercial and industrial loans, project finance, municipal exposures). For these portfolios, top-down models serve better as a secondary or “challenger” modeling approach, rather than a firm’s primary modeling methodology. 

Bottoms-up modeling refers to counterparty or borrower-level analyses. Typically, the risk drivers for a specific segment or industry are correlated to macroeconomic variables.  Granular, borrower-level analysis reaches beyond regulatory mandated stress testing and can serve as a foundation for risk-based pricing, improved budgeting and planning, economic capital modeling and limit- and risk-appetite setting. It can also highlight the most desirable banking relationships while isolating the riskiest relationships and concentrations.   

Methodologically, there are several approaches to bottoms-up modeling. Many banks use actuarial modeling to determine credit risk transition, delinquency, default, as well as loss frequency and magnitude, but often miss critical factors such as the timing of delinquency, default and losses, which requires cash-flow based approaches. Many organizations do not possess the required data necessary to calibrate credit-adjusted, cash-flow models.

Few institutions have systemically collected borrower-level financial statements, or default and loss data over several business cycles. However, many treasury and asset-liability committee (ALCO) members prefer to think of balance sheet risk in a cash-flow (i.e., option-adjusted) fashion.  As a result, many organizations are required to supplement internal modeling with external data, modeling, and model calibration techniques from third parties. 

Bottoms-up modeling for stress testing will soon be applied to Basel III, potentially making it the preferred methodology in the long term.  For bank officers embarking on developing a stress-testing program who are less familiar with data and risk quantification requirements, development and firm-wide adoption may require additional time and cross-organizational buy-in.  Rapid implementation timelines driven by regulation, flexibility and intuitiveness of the approach, may make top-down modeling more attractive in the short-run; however, while expediency to meet requirements is critical, it is equally important to ensure the firm’s modeling architecture is designed to be leveraged and re-used once the firm is ready to graduate to a more comprehensive and holistic approach. 

While no single modeling approach has been blessed by regulators or emerged as a best practice, one thing appears clear: Use of multiple, conceptually sound approaches is prudent given the imprecision of existing state-of-the-art modeling techniques. Regulators also recommend multiple approaches.