To the Editor
Shown are estimated changes in payments per episode of care for five common medical bundles since October 1, 2013. Estimates that were reported in the Bundled Payments for Care Improvement (BPCI) Fourth Annual Evaluation Report2 are for the first 3 years of the model. The estimate for the sample pooled across the five medical bundles is −$353 per episode (P=0.049). Estimates that were reported by Joynt Maddox et al. are for the first 9 months of the model. The estimate for the pooled sample is $112 (P=0.79).
Joynt Maddox et al. (July 19 issue)1 report that hospitals participating in the Bundled Payments for Care Improvement (BPCI) initiative did not reduce Medicare payments in five common medical bundles under BPCI. This finding is inconsistent with the BPCI Fourth Annual Evaluation Report, which showed for each of these five bundles that skilled nursing care was substituted with less expensive home health care, which led to declines in average Medicare payments (Figure 1).2 Although many of these estimates are not statistically significant, the preponderance of evidence suggests that these hospitals were able to reduce Medicare payments by influencing care at post-acute care facilities. A number of factors may explain these differences in the findings. The BPCI analysis covers the first 3 model years (vs. 9 months in the analysis of Joynt Maddox et al.) and is based on episodes that are a (possibly nonrandom) subset of discharges owing to complex attribution rules. Their difference-in-differences estimates may be subject to larger idiosyncratic fluctuations than those in the BPCI report, owing to a shorter baseline period (6 months vs. 1 year). Perhaps most important, their baseline extends into BPCI phase 1, when some hospitals started changing behavior, which may have contaminated their estimates.
Daver C. Kahvecioglu, Ph.D.
Centers for Medicare and Medicaid Services, Baltimore, MD
No potential conflict of interest relevant to this letter was reported.
1. Joynt Maddox KE, Orav EJ, Zheng J, Epstein AM. Evaluation of Medicare’s Bundled Payments initiative for medical conditions. N Engl J Med 2018;379:260–269.
2. Dummit L, Marrufo G, Marshall J, et al. CMS Bundled Payments for Care Improvement initiative models 2-4: Year 4 evaluation and monitoring annual report. Falls Church, VA: Lewin Group, June 2018 (https://innovation.cms.gov/Files/reports/bpci-models2-4-yr4evalrpt.pdf).
We conducted an intention-to-treat analysis that included follow-up data on participants regardless of whether they remained in the program, whereas the Lewin Group conducted a treatment-on-the-treated analysis, which included only data before dropout from the program. This is important given the nearly 50% dropout rate of hospital–condition pairs.1 It seems likely that hospitals that are expecting to do poorly would be more likely to drop out, leading to a possible overestimate of savings in a treatment-on-the-treated analysis, whereas an intention-to-treat analysis might lead to an underestimate of savings. Our sensitivity analysis, which included only hospitals that remained in the program, showed larger although nonsignificant declines in spending at skilled nursing facilities (−$149, P=0.47; vs. $45, P=0.78, in the overall sample). We believe that our findings and those of the Lewin Group on behalf of CMS are complementary and underscore the importance of methodologic decisions in program evaluation.
Karen E. Joynt Maddox, M.D., M.P.H.
Washington University School of Medicine, St. Louis, MO
E. John Orav, Ph.D.
Brigham and Women’s Hospital, Boston, MA
Arnold M. Epstein, M.D., M.A.
Harvard School of Public Health, Boston, MA
Since publication of their article, the authors report no further potential conflict of interest.
1. Joynt Maddox KE, Orav EJ, Zheng J, Epstein AM. Participation and dropout in the Bundled Payments for Care Improvement initiative. JAMA 2018;319:191–193.