They sent chills down the spines of every big bank CEO, forcing some of the most storied names on Wall Street to try multiple times before receiving a pass. Since then, the Federal Reserve’s annual stress tests – aka the Comprehensive Capital Analysis and Review (CCAR) – have become something of a box-ticking exercise. No major bank has failed the capital adequacy test since 2017.
That’s not to say they aren’t useful. Asset managers saw the need to do stress testing as well; despite their flaws, those who used CCAR stress scenarios ended up being better prepared for COVID than their peers. This crisis, of course, is very different from all the others that came before; it didn’t follow any prescribed hypothetical scenarios. Which begs the question: how can you stress test ahead of the time and be ready for the unexpected?
As we’ve discussed before, the only way to cover all possible futures is to have multiple stress scenarios. But it’s more than that. The process of generating multiple stress scenarios is an algorithmic one that should create all kinds of combinations of potential shocks and their consequences. Such models have inputs and assumptions baked in that could be calibrated based on expert outlooks, fundamental views and business expertise (in addition to available historical data).
So how can you make sure that your stress scenarios are good enough to be prepared for a crisis on the COVID scale of magnitude? First, it’s important to remember that crises are (mercifully) rare. Most stress scenarios never materialize. Think of it like fire insurance for your house: you buy it and carry the cost, even though you’re highly unlikely to ever need it (you hope!). Statistically, though, there are enough housefires for insurance companies to estimate probabilities and severities, enabling them to price the product for all properties – even those (the majority) that have never experienced a fire.
Alas, this kind of data doesn’t exist for stress scenarios that are rare and infrequent. To avoid falling into the common trap of only testing yourself on historical scenarios, one needs to come up with a large set of scenarios that include unprecedented ones. But how many is enough? And how can you be sure you’re including scenarios that have never happened before? That’s where back-testing comes in. This means starting before a known stress period – such as the global financial crisis – and then generating forward-looking scenarios to see if the impact on your balance sheet was within the cone of uncertainty you generated (see chart).
The key point in such back-testing is to calibrate your models based on historical data that existed before the stress period started. When you can see that scenarios generated as of July 1, 2019 contain the market gyrations of March and April of 2020, you can be confident your future stress scenarios will cover upcoming, unprecedented stresses as well, no matter the cause