Weighted Noise: Discretion in Regulation

Human discretion is a defining feature of legal and regulatory decision-making. While discretion lets professionals incorporate subtle soft information that rules or algorithms may miss, it also introduces noise— disagreement among decision-makers, such that the same case could yield different outcomes depending on who handles it. We study this trade-off in U.S. bank supervision, where examiners issue CAMELS ratings that shape bank capitalization and lending. Using confidential supervisory data, we model final ratings as weighted sums of component issues. We find that disagreement arises through three channels: (1) examiners treat all issues, even relatively objective ones, as subjective; (2) they disproportionately weight more subjective issues, with management quality alone accounting for nearly 50% of the total weight; and (3) they disagree on the weights applied to each issue, even when agreeing on the issues themselves. We measure discretion as the residual in ratings unexplained by bank fundamentals—this residual captures both soft information and examiner-specific judgment. Because examiner assignment is quasi-random, systematic variation in this residual isolates noise rather than signal. This noise has real consequences: tougher examiners cause banks to raise capital and cut lending for years, and the expectation of unpredictable supervision can prompt banks to preemptively reduce lending. Yet discretion is not purely negative; the discretionary rating predicts future bank deterioration more accurately than a simple model based solely on hard data. Using machine-learning models as a high-dimensional benchmark, we further show that algorithms outperform supervisory ratings in narrowly targeted forecasting tasks but do not replicate supervisory assessments themselves. Nonetheless, excessive discretion adds noise without extra signal. Moderately constraining how subjective components are weighted improves predictive accuracy and reduces costly disagreement.