Are Hospital Quality Indicators Causal?
Hospitals play a key role in patient outcomes and spending, but efforts to improve their quality are hindered because we do not know whether hospital quality indicators are causal or biased. We evaluate the validity of commonly used quality indicators, such as mortality, readmissions, inpatient costs, and length-of-stay, using a quasi-experimental design where hospital closures reallocate large numbers of patients to hospitals of different quality. This setting allows us to measure whether patient outcomes improve as much as quality indicators predict when a relatively low-quality hospital closes, or decline as predicted when a relatively high-quality hospital closes. Using more than 20 years of Medicare claims for over 30 million patients admitted with five common diagnoses, we find that hospital quality indicators overstate differences in the causal impact of hospitals on mortality and readmission rates by 7 percent or less, but overstate differences in the causal impact of hospitals on inpatient cost and length-of-stay measures by closer to 40 percent. On average, hospital closures reduce patient mortality by shifting patients to higher quality hospitals, but the but the effect varies widely depending on the relative quality of the closing hospital.
We are grateful to Jon Skinner and participants at various seminars for many helpful comments. This research was supported by the NIH through a Pilot grant and a P01 grant to Dartmouth and the NBER (P30AG012810 and P01AG019783). The programs used in the paper can be requested from the authors. We are also able to share hospital-level quality measures, provided it meets CMS’ requirements for data-release. Staiger is co-founder and has an equity interest in ArborMetrix, a company that provides performance measurement systems in healthcare. Chandra serves on the Congressional Budget Office’s Panel of Health Advisors. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
Douglas O. Staiger
Douglas Staiger is a co-founder, consults for, and holds an equity interest in ArborMetrix, Inc., a company that sells efficiency measurement systems and consulting services to insurers and hospitals.