Observational Intensity Bias Associated with Illness Adjustment: Cross Sectional Analysis of Insurance Claims
John E Wennberg, professor emeritus in the evaluative clinical sciences, Douglas O Staiger, professor of economics, Sandra M Sharp, research associate, Daniel J Gottlieb, research associate, Gwyn Bevan, professor of policy analysis, head of the department of management, Klim McPherson, visiting professor of public health epidemiology and emeritus fellow, H Gilbert Welch, professor of medicine, professor of The Dartmouth Institute, and professor of community and family medicine
Abstract
Objective To determine the bias associated with frequency of visits by physicians in adjusting for illness, using diagnoses recorded in administrative databases.
Setting Claims data from the US Medicare program for services provided in 2007 among 306 US hospital referral regions.
Design Cross sectional analysis.
Participants 20% sample of fee for service Medicare beneficiaries residing in the United States in 2007 (n=5 153 877).
Main outcome measures The effect of illness adjustment on regional mortality and spending rates using standard and visit corrected illness methods for adjustment. The standard method adjusts using comorbidity measures based on diagnoses listed in administrative databases; the modified method corrects these measures for the frequency of visits by physicians. Three conventions for measuring comorbidity are used: the Charlson comorbidity index, Iezzoni chronic conditions, and hierarchical condition categories risk scores.
Results The visit corrected Charlson comorbidity index explained more of the variation in age, sex, and race mortality across the 306 hospital referral regions than did the standard index (R2=0.21 v 0.11, P<0.001) and, compared with sex and race adjusted mortality, reduced regional variation, whereas adjustment using the standard Charlson comorbidity index increased it. Although visit corrected and age, sex, and race adjusted mortality rates were similar in hospital referral regions with the highest and lowest fifths of visits, adjustment using the standard index resulted in a rate that was 18% lower in the highest fifth (46.4 v 56.3 deaths per 1000, P<0.001). Age, sex, and race adjusted spending as well as visit corrected spending was more than 30% greater in the highest fifth of visits than in the lowest fifth, but only 12% greater after adjustment using the standard index. Similar results were obtained using the Iezzoni and the hierarchical condition categories conventions for measuring comorbidity.
Conclusion The rates of visits by physicians introduce substantial bias when regional mortality and spending rates are adjusted for illness using comorbidity measures based on the observed number of diagnoses recorded in Medicare’s administrative database. Adjusting without correction for regional variation in visit rates tends to make regions with high rates of visits seem to have lower mortality and lower costs, and vice versa. Visit corrected comorbidity measures better explain variation in age, sex, and race mortality than observed measures, and reduce observational intensity bias.
