Bank Stress Test Severity: Perception vs. Reality

14 Nov 2018

A research note written by Sean D. Campbell, Executive Vice President, Director of Policy Research, Financial Services Forum

Download as a PDF..

Background

Each year, the Federal Reserve conducts a stress test of the nation’s largest banks to assess their capital adequacy.  To do this, the Fed specifies a severely adverse stress scenario that is comprised of the 13-quarter path of 28 macroeconomic and financial variables ranging from unemployment and real GDP to exchange rates.  A key public policy issue is the degree of severity embedded in the Fed’s scenario.  A more severe scenario will imply larger hypothetical losses in the stress test, which, in turn, will require banks to maintain higher capital levels.  As capital is an expensive form of funding, higher capital requirements can result in higher borrowing costs and reduced economic growth.  Accordingly, the severity of the stress scenario represents an important public policy choice that should be made in a transparent and objective manner.

In 2013, the Fed issued a scenario design framework that lays out its approach to specifying stress scenarios.  The framework is characterized as a “recession approach” in which the scenario variables are specified to “resemble the paths of those variables observed during a recession.”  The framework also makes clear that the Fed “intends to use the unemployment rate as the primary basis for specifying the severely adverse scenario.”  Further, the framework specifies that “all other variables in the severely adverse scenario will be specified to be consistent with the increase in the unemployment rate.”

In this post, we examine how the path of GDP, the single most comprehensive measure of economic activity, is specified in the stress test. In particular, we examine how the Fed’s calibration of GDP in the stress test compares with historical recession experience, the approach taken by the Bank of England (BOE) in its stress tests, and the implications of the Fed’s approach to GDP for the severity of the stress test.

Okun’s Law: Historical Recession Data and the Fed’s Scenario Assumptions

Figure 1 shows four scatterplots.  Each scatterplot also shows a superimposed regression line, the estimated slope of the regression line, and the R-squared of the regression, which measures the degree of association between each of the variables in the scatterplot. An R-squared value of 100% implies a perfect linear relationship between two variables.

Sources: Federal Reserve and Bank of England. See Appendix

The scatterplot in the top left corner shows the empirical relationship between quarterly unemployment rate changes and quarterly real GDP growth during U.S. recessions that are considered in the Fed’s scenario design framework.  The top right corner scatterplot shows the empirical relationship between the Fed’s projected path for quarterly unemployment rate changes and quarterly GDP growth as specified in its stress test scenarios between 2011 and 2018.[1]  Essentially, the left-hand plot shows how unemployment and GDP data are actually related during historical recessions and the right-hand plot shows how the Fed assumes these variables are related in its stress scenarios.

Both scatterplots in the top row of Figure 1 show a clear and inverse relationship between unemployment and GDP.  This result is not surprising because this relationship, known as Okun’s Law, is a well-known and robust empirical feature of the data that has been rigorously documented since the 1960s.  However, both scatterplots also reveal striking differences between the data and the Fed’s scenario assumptions.

First, while the relationship between unemployment and GDP in the historical data is highly imperfect, the Fed’s approach leads to a nearly perfect relationship as gauged by the difference in the reported R-squared values (33% vs. 96%).  Under the Fed scenarios, there is little room for any other factors beyond unemployment to influence GDP.  The data, however, show that a host of additional factors influence GDP in recessions.  This is important as it suggests that the Fed’s scenario framework may focus too narrowly on unemployment at the expense of other important factors, such as financial conditions, that should be considered when specifying the path for GDP.

Second, Figure 1 also shows that the Fed assumes a larger response of GDP to unemployment than is found in the data.  In particular, the Fed assumes that a one percentage point increase in unemployment lowers GDP by 1.93 percentage points (Slope: -1.93).  The historical data, however, indicate that during recessions a one percentage point increase in unemployment lowers GDP by 1.31 percent (Slope: -1.31).  The Fed’s assumed increased sensitivity of GDP to unemployment is important because it will imply even steeper declines in GDP than would be indicated by historical recession data.

The bottom row of scatterplots in Figure 1 repeats the same analysis presented in the top row using data from the U.K. and the BOE’s stress test scenarios.  Unlike the Fed, the BOE’s stress scenarios contemplate a wider range of factors affecting real GDP as the R-squared in the bottom right corner scatterplot is only 79% rather than 96%.  Also, the BOE’s assumed sensitivity of GDP to unemployment is more in line with historical recession experience as the estimated slope coefficient is roughly -1.3 in both the BOE’s assumptions and the U.K. data.  Accordingly, these results suggest that the BOE’s assumptions regarding GDP and unemployment comport more closely with historical recession experience than the Fed’s assumptions.

In Figure 2 we show the Fed’s assumed path for GDP in the 2018 stress test (blue) versus a sensitivity adjusted path (orange) that adjusts the Fed’s path by the degree to which the Fed’s Okun’s Law assumptions are more extreme than the relationship found in the historical data and presented in the top-left panel of Figure 1.  Figure 2 also shows the path of GDP from the 2007-2009 recession for comparison (grey).  The 2018 Fed scenario implies a much steeper decline in GDP than was observed during the 2007-2009 recession.  Figure 2 also shows that much of the 2018 scenario decline owes to the assumed oversensitivity of GDP to unemployment as the sensitivity adjusted path is much shallower and more consistent with the 2007-2009 recession’s GDP path.

Source: Federal Reserve, See Appendix

This finding is important for two reasons.  First, it suggests that the path of a key macroeconomic variable in the stress test is not consistent with historical recessions as required by the Fed’s scenario design framework.  GDP reacts more strongly to unemployment in the Fed’s scenarios than is indicated by historical experience during recessions.  Second, it shows that gauging the severity of the stress scenario by the path for unemployment alone is inadequate since other key variables, such as real GDP, may take extreme paths that further compound stress test losses and imply even higher required capital levels than would be suggested by the path of unemployment alone.

Gauging Impact: Oversensitivity and Stress Test Losses

So, if GDP declines are overstated in the Fed scenarios, how much does this increase stress test losses and raise capital requirements?  Unfortunately, the lack of transparency of the Fed stress tests makes it impossible to answer this question precisely.

While it is impossible to answer this question precisely, it is possible to construct an estimate.  Specifically, GDP can be empirically related to aggregate data on bank revenues and losses to get a sense of how sensitive bank revenues and losses are to changes in GDP.  In Figure 3 we present a time-series plot of aggregate bank return on assets (ROA) – a summary measure of revenues and losses – and the predicted ROA that results from estimating an empirical model for ROA that includes GDP.  As the plot shows, while the model’s fit is imperfect, it is able to capture many of the pronounced swings in ROA, including the steep decline observed during 2007-2009.  This result also makes intuitive sense because GDP is a broad and comprehensive measure of economic activity and bank revenues and losses would be expected to decline when economic activity declines.

Sources: FRBNY and FRED, See Appendix

Using this simple model, the difference in ROA is computed assuming two paths for real GDP: (1) the path from the 2018 stress scenario (blue) and the sensitivity adjusted real GDP path (orange) presented in Figure 2.  According to this model, predicted ROA from the sensitivity adjusted path consistent with historical recession experience is roughly 75% higher than the predicted ROA using the Fed’s 2018 scenario.  This finding suggests that the steep declines in real GDP assumed by the Fed can have material consequences for the stress tests and resulting capital requirements.  Moreover, to the extent that these paths do not comport with the behavior of GDP during recessions, these steeper GDP declines may not be consistent with the Fed’s own scenario design framework.

Conclusion

While the Fed has indicated that it calibrates the stress scenario consistent with historical recessions, we have provided evidence to the contrary.  Specifically, we find that the Fed’s modeling assumptions rely too heavily on unemployment as a driver of GDP and imply a greater degree of sensitivity of GDP to unemployment than found in historical recession data.  This results in greater GDP declines in stress scenarios that likely imply larger stress test losses.  We also find that the Fed’s assumptions regarding GDP are out of line relative to the BOE’s GDP assumptions.

In light of these findings, the Fed should consider clarifying to the public, with considerably more specificity, how it calibrates its stress scenarios.  The Fed should also consider releasing a comprehensive and quantitative measure of scenario sensitivity that considers the combined impact of all macroeconomic variables that are projected in its stress test.  Such a measure would allow the Fed to credibly explain the degree of severity chosen and why a given level of severity was chosen.  The Fed should also consider whether its assumed relationship between unemployment and GDP is consistent with its own scenario design framework.  Finally, greater transparency from the Fed, on both scenario severity and the specification of its stress testing models, would improve the important public policy discussion about stress testing and bank capital.  Given that the Fed’s estimated stress losses translate directly into capital requirements, which impact the cost of borrowing for businesses and households, it is imperative that the public have a clear and comprehensive view of stress test severity.

[1] The Fed has conducted stress tests every year since 2010 but in 2010 it did not disclose the numerical paths for real GDP and unemployment but only provided a graphic plotting these variables.  Accordingly, we omit the real GDP and unemployment paths from the 2010 stress test.

 

 

Learn More:

Forum Statement on Stress Test Results

“The CCAR results again demonstrate that the nation’s largest financial institutions are strong, resilient, and positioned to promote economic growth and job creation even in times of severe financial stress.” Read the full statement.