Testing macroprudential stress tests: The risk of regulatory risk weights

•We compare the outcomes of regulatory stress tests to those of a methodology based on market data.•We find differences in the rankings of banks by their required capitalization.•Differences arise due to the reliance on risk-weighted assets in regulatory capital requirements.•Results indicate that regulatory stress tests could be more effective.•In particular, using complementary capital adequacy definitions based on total assets and market risks.

We compare the capital shortfall measured by regulatory stress tests, to that of a benchmark methodology — the “V-Lab stress test” — that employs only publicly available market data. We find that when capital shortfalls are measured relative to risk-weighted assets, the ranking of financial institutions is not well correlated to the ranking of the V-Lab stress test, whereas rank correlations increase when required capitalization is a function of total assets. We show that the risk measures used in risk-weighted assets are cross-sectionally uncorrelated with market measures of risk, as they do not account for the “risk that risk will change.” Furthermore, the banks that appeared to be best capitalized relative to risk-weighted assets were no better than the rest when the European economy deteriorated into the sovereign debt crisis in 2011.