Stress Testing 101: The method, recent developments, transparency concerns and some recommendations
|
||||||||||||||||||||||||||||||||||||
| Table 1: Scenario stress test approaches | |
Portfolio-driven approach |
Event-driven approach |
Risk managers in a firm initially discuss and identify the vulnerabilities in the portfolio currently held by the firm. After determining these vulnerabilities, risk managers work backwards and formulate scenarios under which these vulnerabilities are stressed. |
The scenario is formulated based on events and how these events might affect the relevant risk factors in a firm’s portfolio. These scenarios are often formulated at the request of senior management or the board. |
Figure 1: Portfolio-driven approach

Figure 2: Event-driven approach

Furthermore, scenario tests can be categorised as either historical or hypothetical scenarios. The differences between these two categories are explained in table 2.
| Table 2: Historical vs. Hypothetical scenarios. | |
Historical scenarios |
Hypothetical scenarios |
Rely on a significant market event experienced in the past. |
A significant market event that has not yet happened. |
This type of scenario tends to be more fully articulated as they reflect an actual stressed market environment and therefore involve fewer judgments. However, these scenarios may not reflect the new ways in which financial risk is being packaged. |
These scenarios are potentially more relevant to the risk profile of the firm. However, these scenarios are more labour-intensive and involve considerably more judgment. |
It should also be noted that Hybrid scenarios are quite common, and are hypothetical scenarios which are informed by historical market moves, but which are not necessarily linked to a specific crisis. These types of scenarios thus use historical episodes to assist in the calibration of hard-to-set factors.
2. Sensitivity (stress) tests
Sensitivity type stress tests are the most straightforward type of test and are a simple (sensitivity) test whereby risk parameters are moved instantaneously by a unit amount, such as a 10% increase of a 10 basis point decline.
These tests can be run relatively quickly and are used by the board and senior management to form a first approximation of the impact on the firm of a move in a financial variable. These tests are also widely used at trading desks and business (unit) level.
Micro & Macro stress tests
Stress tests are also used at a higher level, These types of stress tests can be divided into (1) micro stress tests, which are run by individual financial institutions for the purpose of the institution’s risk management, and (2) macro stress tests, which assess the resilience of the financial system as a whole.
| Table 3: Micro & Macro stress tests. | |
Micro stress tests |
Macro stress tests |
Run by individual financial institutions for risk management purposes. These types of tests often ignore behaviour of competitors. The institution’s balance sheet and income composition affect the response to shocks. |
Run by central banks / regulatory authorities. These types of tests assess the resilience of the financial system as a whole. Macro stress testing is a tool for assessing the vulnerability of the financial system to potential macroeconomic shocks. |
According to the BIS there are two defining elements of a macroprudential orientation. The first defining element is a focus on the financial system as a whole as opposed to individual financial institutions. This macroprudential orientation highlights the cost to the macroeconomy of stress in the financial system in a general manner. The second defining element is an emphasis on the dependency of aggregate risk on the collective behaviour of individual institutions – often called “endogeneity of risk”. A microprudential orientation puts more emphasis on the losses incurred by individual financial institutions per se.
Table 4 presents additional differences between the focus areas of macro and micro stress tests.
| Table 4: Micro & Macro stress test focus area differences. | |
Micro stress tests |
Macro stress tests |
Tends to take movements in asset prices and the macroeconomic backdrop as given – as “exogenous”. |
Highlights the fact that asset prices and the macroeconomy are themselves strongly affected by how financial institutions behave. |
It is also important to note that macroeconomic stress testing differs from portfolio stress tests, due to aggregation and also in terms of the objective.
Portfolio stress tests are used to determine whether the amount of risk inherent in a portfolio is justified by the expected return, while aggregate stress testing is used to measure structural vulnerabilities(4) and the risk situation in the entire financial system.
Aggregate stress tests assess the impact of potentially adverse events on the entire financial system and provide policymakers with the option to take measures before a crisis develops.
Macroeconomics stress testing approaches
There are two main approaches to conducting stress tests on an aggregate level.
The first approach is to use the individual stress tests conducted by financial institutions’ risk management teams and to simply add up the results to obtain an aggregate stress test result. This approach, however, requires consistent stress testing methodologies to be applied by all financial institutions and the same stress test scenario must be applied across all the institutions.
Developing a common risk scenario can be problematic as different institutions have different portfolio compositions with different exposures to risk. Some banks, for example, may have higher exposure to one specific country or region since it engages in lending in these foreign currencies. Other banks may have a strong focus on domestic issues or on the housing market via relatively large mortgage loans.
The second approach is for a supervisory or regulatory body to first aggregate portfolio and balance sheet data from individual financial institutions and then to conduct stress tests on this aggregated data. This approach requires the regulatory body to obtain the relevant raw data from the individual financial institutions, and that the institutions follow the same reporting and accounting guidelines. This is the case in order for the data to allow comparison and aggregation.
Performance of stress testing during the crisis
According the BCBS, the financial crisis has highlighted weaknesses in stress testing practices employed prior to the start of the crisis in four broad areas(5):
Table 5 provides some details on each of the four broad areas in which weaknesses have been identified and improvement is required.
| Table 5: Identified stress testing weaknesses and required improvements.(6) |
(i) use of stress testing and integration in risk governance |
Further investments in information technology infrastructure may be necessary to enhance the availability and granularity of risk information that will enable timely analysis and assessment of the impact of new stress scenarios designed to address a rapidly changing environment. |
(ii) stress testing methodologies |
|
(iii) scenario selection |
Historical scenarios were often implemented based on a significant market event experienced in the past – such stress tests were not able to capture risk in new products that have been at the centre of the crisis. In addition, the severity levels and duration of stress indicated by previous events proved to be inadequate – this thus resulted in the historically based stress tests underestimating the level of risk and also the interaction between risks. Hypothetical stress tests – these tests aim to capture events that had not yet been experienced. Prior to the crisis, banks generally applied only moderate scenarios, either in terms of severity or the degree of interaction across portfolios or risk types. Scenarios that were considered extreme or innovative were often regarded as implausible by the board and senior management. |
(iv) stress testing of specific risks and products |
Particular risks that were not covered in sufficient detail in most stress testing include:
Scenarios were not sufficiently severe when stress testing structured products and leveraged lending prior to the crisis – to some degree this may be attributed to reliance on historical data.
Overall, stress tests of structured products suffered in that they failed to recognise that risk dynamics for structured products are different from those of similarly-rated cash instruments. These differences were even more pronounced during the crisis, further degrading the performance of stress tests. Stress tests should specifically consider the credit quality of the underlying exposures, as well as the unique characteristics of structured products. In addition, stress tests also assumed that structured product markets would remain liquid and if market liquidity would be impaired it would only be the case for short periods – this meant that banks underestimated the securitisation risk related to issuing new structured products.
Stress testing also only dealt with directional risk and did not adequately capture basis risk, thereby reducing the effectiveness of hedges. The crisis also featured greater wrong-way risk(9).
Stress testing for counterparty credit risk typically only stressed a single risk factor for a counterparty, they were also insufficiently severe and usually omitted the interaction between credit risk and market risk(10). The crisis showed that stress testing for counterparty credit risk should be improved by using stresses applied across counterparties and to multiple risk factors, as well as those that incorporate current valuation adjustments.
Yet another model weakness that was identified was that they did not adequately capture contingent risks that arose either from legally binding credit and liquidity lines or from reputational concerns to for example off-balance sheet vehicles.
Stress tests did not truly capture the systemic nature of the crisis or the magnitude and duration of the disruption to interbank markets. |
Changes in stress testing practices since the outbreak of the financial crisis
Given the unexpected severity of events, stress testing has gained greater prominence within banks as a complementary risk management and also capital planning tool (to provide an altered risk perspective).
That said, banks recognise that current stress testing frameworks must be enhanced both in terms of granularity of risk representation and the range or risks considered. Several banks have begun to address these issues and other weaknesses of stress tests for the specific risks mentioned. More general areas in which banks are considering future improvement include:
Firm-wide stress testing is also an area that many banks recognise that improvement is required in order to ensure appropriate risk capture and also to aggregate risk more effectively across business lines(11).
After the onset of the crisis, ad hoc “hot-spot” stress testing has been employed by some banks as a tool to inform senior management’s crisis management decisions. The ability to conduct stress test at very short notice has proven to be valuable during a period of rapidly changing market conditions.
Recent (global market) developments concerning stress tests
By opening the financial sections of most newspapers, readers would recently have come across a number of news articles pertaining to stress testing and opinions and analysis – specifically on how stress tests are impacting the current and possibly the future of financial institutions and regional financial systems. It is of interest if not scary to see the fragile state of many financial institutions as many of them fail to meet the stress testing requirements.
Before delving into some recent developments and impacts concerning stress testing it is important to note that the Basel Committee for Banking Supervision (BCBS) in May 2009 released a document entitled, ‘Principles for sound stress testing practices and supervision’ after engaging with the industry in examining stress testing practices. The paper assesses stress testing practices during the crisis and based on this assessment and in an effort to improve practices, the BCBS (through this particular paper) develops sound principles for banks and supervisors in terms of stress testing. These principles cover the overall objectives, governance, design and also implementation of stress testing programmes while also touching on issues related to stress testing of individual risks and products. The papers’ recommendations are aimed at deepening and strengthening banks’ stress testing practices and supervisory assessment of these practices.
The following extracts provide an indication of both the impact and the heightened role stress tests are currently playing throughout different financial jurisdictions and sectors.
During March, all but four of 19 U.S. banks got the green light from the Federal Reserve, which declared them strong enough to survive a downturn worse than the Great Recession. With this the Fed’s findings signalled its confidence that the financial system is healthy again after it nearly collapsed 3-and-a-half years ago.
“It’s clearly good news – the U.S. banking system can now withstand a quite severe recession without falling over” said Douglas Elliot, a fellow at the Brookings Institute(12).
The Fed reviewed the balance sheets of 19 bank holding companies to determine whether they could withstand a severe crisis: unemployment at 13 percent, stock prices falling to 60 percent over two years and a home price decrease of 21 percent from current levels.
J.P. Morgan Chase and Wells Fargo were amongst the large bank holding companies that passed the Fed’s stress test, but Citigroup (the third-largest bank in the U.S.) was one notable exception that failed the test. Citigroup was amongst the companies that lacked enough capital to withstand another severe economic and financial crisis where after its stock price fell 3 percent in after-hours trading(13).
The other three financial institutions that did not pass the Fed’s hypothetical stress tests were Ally Financial(14), SunTrust and MetLife.The banks that passed the stress tests celebrated with announcements of increased dividends and stock buy-back plans, while the Fed can bar the banks that failed the test from paying (or increasing) dividends or buying back their own stock. The Fed can also force them to raise money by selling additional stock or issuing debt, although the Fed did not order any of the banks that failed to raise specific funds. The Fed did, however, require the ‘failed’ banks to submit plans outlining how they plan to get stronger within 30 days.
The Fed has conducted the stress test each year since 2009, however, the test results were not released in 2010 or 2011. After the tests in 2009, the Fed ordered 10 banks to raise a total of $75 billion with the Bank of America alone to raise $34 billion. The 2012 test was more rigorous, according to the Fed, so it could be assured that the industry was prepared to meet more stringent international banking rules that go into effect in 2013 – not only does the Fed want banks to show they could withstand the crisis but also keep lending.
In terms of the impact of real estate, some analysts noted that the Fed’s tests don’t adequately capture all of the risks banks still face from bad real estate investment on their books.
A report by The West Australian during the month of June stated that only 70 percent of Spanish banks will pass the IMF’s stress tests. This is probably mostly as a result of bad real estate loans following the 2008 collapse of the property bubble. Bankia is a big part of the not so successful 30 percent and asked the state for the biggest rescue in Spanish banking history (€19 billion), in addition to the €4.5 billion already received.
Moody’s cut the debt ratings of 16 Spanish banks, on 31 May 2012, by one to three notches, citing the effects of the ongoing recession and the Spanish government’s own reduced creditworthiness. The leading two Spanish banks (Santander and BBVA) were both hit with three-notch downgrades (from Aa3 to A3).
In the US a separate round of stress tests for medium-sized banks could cause some pain for investors and five specific banks are potentially “more at risk” than others(15). The Federal Reserve has already been conducting annual stress tests for bank holding companies with total assets of more than $50 billion. The Fed, along with the Federal Deposit Insurance Corporation (FDIC) and the Office of the Comptroller of the Currency, will require banks with over $10 billion in total assets to stress test themselves, “in a broad range of scenarios from an individual business-line to enterprise-wide perspective”. Banks are required to hand in their stress test result to the primary regulatory by 5 January 2013.
An analysis by Sterne Agee states that “banks closer to $50 billion with high payout ratios, low profitability levels, low capital levels, outsized concentrations, high ratios of problem loans to capital and reserves and high loan/deposit ratios” are more at risk for regulatory action from the stress test than those closer to the $10 billion threshold.
During July the European Banking Authority (EBA)(16) reported that Europe’s major banks complied with demands to bolster their reserves, but only by relying heavily on government aid, accounting techniques, asset sales and methods other than raising cash from investors.
The EBA’s new data underscored the troubles still facing Europe’s financial system as banks cope with an economic recession, a growing number of bad loans and a government debt crisis that has undercut the value of bond holdings central to many bank investment portfolios. This may also form part of the debate underway in Europe over how to create a regional system for regulating banks in order to stabilise the financial system as a whole. The London-based EBA may play a role in that new regulatory structure, but its handling of bank stress tests during 2011 and its effort to pump up banks’ capital reserves have been widely criticised.Several of the banks that were ordered to raise fresh reserves during the latter part of 2011 have since failed, and others have needed far more funds than the EBA ordered. For example, the Spanish bank Bankia, was estimated by the EBA to need less than $2 billion, but Bankia has since requested more than $20 billion from the Spanish government – which partly resulted in the country to seek outside aid.
In essence the EBA is an authority which coordinates the work of financial regulators throughout the European Union, but it does not directly supervise banks as it is tasked (by the European finance ministers) to develop stress tests to evaluate the strength of the region’s financial system and oversee the effort to raise bank capital levels.In addition, the EBA officials stated that this last report may be flawed, as a major disclaimer in the document cautioned that in analysing each bank, “no assessment of assets quality was undertaken”. The EBA had thus taken each bank’s word for the value of its investments.
To conclude, the recent events have shown just how much the EBA’s analysis didn't capture. For instance, little of the $120 billion requested by Spain from the European bailout fund to rehabilitate its financial system was reflected in the various EBA stress tests or the capital increases. This is because the institutions involved were smaller savings banks that were not included in the EBA analyses.
Earlier this year Fitch Ratings reported that UK credit card master trusts remain resilient against a hypothetical severe economic deterioration in three stress tests, the agency conducted. The three scenarios demonstrated the immediate stress required to see multi-category downgrades of Fitch’s UK credit card ratings. The stress tests only tested the impact on the rating should the performance (in terms of change), monthly payment rate (MPR) and yield of the credit card receivables deteriorate, although sovereign, counterparty risk and transaction specific features also influence rating stability.
The three scenarios used by Fitch Ratings are as follows:
The second severe scenario incorporated a 100% purchase rate stress, significantly reducing the repayment speed of credit card receivables. This scenario can also be connected to the originator defaulting and the credit card portfolio being wound down, rather than the business being continued by a successor purchaser. Fitch considered a performance stress similar to the moderate stress, and as a result two ‘AAA’ notes would be downgraded to sub-investment grade with all other ‘AAA’ notes remaining investment grade.
It is thus clear from these limited extracts that stress testing is definitely making headlines, be it for good or bad reasons. In this case bad news can be perceived in a positive light as it results in the continuous improvement of stress testing methods, and the strength of both individual financial institutions and ultimately the financial system.
The transparency and disclosure of stress testing results
One of the main focus areas recently has been on financial stability, primarily due to the recent financial crisis. One of the areas that concerns financial stability is whether or not stress-test and the stress-test results should be made public. This section explores this somewhat new area of focus a bit further.
Public availability of stress test results
The first critical question is whether stress test results should be made public, and if so, at what level of detail. The short answer is that regulators should be mindful of significant pitfalls.
There is a compelling case for publicly releasing stress test results, as disclosure would enhance market discipline by allowing outsiders to better price a bank’s risks. In turn, this would prevent bank insiders from engaging in excessive risk-taking behaviour. Disclosure would also improve financial stability by reassuring outsiders that systemically important banks or financial institutions are adequately capitalised and have the ability to survive a future crisis. Lastly, it would increase the discipline of regulators by exposing them to greater external scrutiny.
However, these benefits, may not be fully realised if stress-test results are not handled properly. Banks have dense and obscure portfolios whose underlying risks are hard to evaluate – simply disclosing the performance of these portfolios under adverse scenarios without any understanding of the underlying risk exposures of the bank may induce perverse incentives. This is so, as a bank’s insiders could choose inefficient portfolios to maximise the probability of passing stress tests rather than maximising the bank’s long-term value.
This section continues with some recommendations concerning stress-testing disclosures in three specific areas which might prove as pitfalls.
Recommendations
The following represents some recommendations to minimise such incentives (as mentioned above):
For market disclosure to work (effectively), stress test results should be accompanied by detailed disclosures of a bank’s exposure by asset class, country and maturity.
If such detailed disclosure is not feasible, it might be recommended to be better not to release the results of individual banks, but only to disclose results aggregated across banks. This of course would mean that the benefit of market discipline would be lost, however, aggregated disclosures would minimise perverse incentives while at the same time still providing the benefit of financial stability.
In addition, the perverse incentives could also be mitigated by not revealing the stress-test model used by the regulator to the bank’s insiders – not knowing how they (the bank) will be graded would make it harder for insiders to manipulate the stress- test.
From a different viewpoint, disclosure may also harm market discipline by causing market participants to overreact to bad news. After all, banks operate in inherently fragile environments which are prone to contagion in the sense that if some creditors lose confidence and “run” others might do so as well. In such environments, disclosure of stress-test results may serve as a signal that coordinates the actions of all the market participants.
Consider for example, if a market participant thinks that after learning the bank’s poor grade other market participants might withdraw their money from the bank, the particular participant might also withdraw his money. If all market participants think the same way about the other participants, a “run” becomes self fulfilling. Note that this overreaction wouldn't occur if the stress test perfectly reflected the bank’s conditions. In such a case, market participants would not care about the behaviour of others as they would know exactly how to react to the bank’s poor grade. Thus, the potential for overreaction to the disclosure only occurs because stress-tests are inherently imperfect measurements of a bank’s condition.
One recommendation which might serve to alleviate such overreaction is that regulators complement the disclosure (of any bad news) with a description of the corrective actions to be taken so that the corresponding role of disclosure is minimised and panic is left un-triggered.
Individual bank results should also only be disclosed when the results are sufficiently precise and reliable. If a situation where this is not the case, the regulator should disclose only aggregated results across banks, which have the advantage of being less likely to be wrong as personal characteristic errors are averaged away in estimating bank conditions. This recommendation is a viable solution if the goal is to promote financial stability.
Disclosure can harm market discipline by reducing the incentives of market participants to produce private information. Essentially regulators rely on two sources of information in determining the financial condition of a bank: (1) the data observed from the market price of a bank’s security, and (2) the data collected by the regulator during the supervisory process.
The market price of a bank’s security aggregates the information of market participants who have monetary incentives to trade on their information and opinions. Disclosing stress-test results might adversely reduce the incentives of market participants as they lose some of their informational advantage and therefore produce less information. This results in a degrading of the information role of the share price and impedes the ability of the regulator to learn from market prices.
A recommendation to such a scenario would be to disclose aggregated information to stress-tests, as such disclosures still convey the benefit of financial stability but does not drown out signals about individual banks from the market participants.
The concluding verdict
The conclusion to this article is short and direct - stress testing is a robust and relevant practice which should be used not only passively but also proactively. In addition, the stress tests required by regulatory authorities were heavily influenced by the good market conditions in the past, now the same must apply concerning the recent market conditions. This will ensure the continuous improvement of the method and also strengthen the financial soundness of individual financial institutions and the financial system as a whole. Lastly, the transparency debate concerning stress tests and stress test results is an area that compels the required attention.
(1) BCBS. 2009. Principles for sound stress testing practices and supervision. Basel, BIS.
(2) Pillar 1 of the Basel II framework focuses on the minimum capital requirements.
(3) BCBS. 2009. Principles for sound stress testing practices and supervision. Basel, BIS.
(4) Vulnerabilities which may lead to massive turmoil in financial markets.
(5) BCBS. 2009. Principles for sound stress testing practices and supervision. Basel, BIS.
(6) BCBS. 2009. Principles for sound stress testing practices and supervision. Basel, BIS.
(7) from individual instrument level up to the institutional level.
(8) such as correlations.
(9) Wrong way risk occurs when exposure to a counterparty is adversely correlated with the credit quality of that counterparty.
(10) Specific wrong-way risk.
(11) It terms of this required improvement it is important that banks not be restricted to a checklist approach even though the BCBS provides principles – these principles are intended to support and reinforce (only).
(12) Brookings Institute is a non-partisan policy think tank.
(13) The Fed released the results after markets had closed.
(14) Formerly GMAC Bank – was the worst performing bank in the Fed’s test.
(15) Five impacted banks include: TCF Financial, Astoria Financial, New York Community Bancorp, Synovus Financial, Valley National Bancorp.
(16) The EBA coordinates the work of financial regulators throughout the European Union.
:: Back
to top :: |