Is Mandatory Participation in Medicare Demonstrations Necessary?

Recently, Health Affairs Forefront, published a post by Dan L. Crippen, former director of the Congressional Budget Office and currently a Healthcare Leaddership Council consultant, that should be a catalyst for discussion on a critical element of the Center for Medicare and Medicaid Innovation’s future direction.

In his post, Dr. Crippen enters the debate over whether models being tested by CMMI should have mandatory or voluntary participation on the part of healthcare providers.  Some have argued that demonstration projects have floundered under voluntary participation because providers have brought in cohorts of comparatively healthy patients not reflective of the Medicare beneficiary population at large. He points to several examples, though, to make the case that voluntary participation did not result in adverse selection and that a more weighty problem plaguing CMMI demonstration projects has been the lack of timely data flowing to model participants.

The Crippen post is below and at the link above, which will take readers to the Health Affairs Forefront site.

***

The tenth anniversary of the Center for Medicare and Medicaid Innovation (the Innovation Center) was in 2020. This anniversary was accompanied by several retrospectives of the results of the Innovation Center’s first decade of operation. Unfortunately, most of the analysts, including those from the Centers for Medicare and Medicaid Services (CMS), reached similar conclusions: that the demonstrations deployed by the Innovation Center neither saved much money nor greatly improved quality, the two primary objectives set out for the Innovation Center in the Affordable Care Act.

Past and present Innovation Center directors concluded that the primary reason for the demonstrations’ failure to achieve the objectives was selection bias by the providers who had volunteered to participate in the various models. The claim is that the providers brought with them a cohort of healthier-than-average patients, making it easy to show savings relative to the benchmark. Relative to providers with sicker patient populations, these providers were more motivated to participate in the demonstrations due to the potential opportunity to earn a bonus from the Innovation Center if they spent less than the Innovations Center’s benchmarks.

Some of the demonstrations included an alignment algorithm for assigning patients to providers within an accountable care organization (ACO), at least one of which assigned patients depending upon their previous use of participating providers in the ACO. In some demonstrations, there was considerable turnover in both the beneficiaries and providers, which theoretically allowed ACOs the opportunity to alter their risk pool by selecting or changing providers (or other aspects of the model) to create a patient population with certain characteristics or health care needs. The former and current directors concluded that only mandatory participation by providers would overcome this perceived selection bias.

However, before seeking a solution to this problem, the question of whether selection by voluntary providers contributed to the disappointing results of the demonstrations should be explored. This article summarizes a multitude of analyses surrounding the reasons the demonstrations show little savings or quality improvement. The analyses indicate that the failure was not due to voluntary, as opposed to mandatory, participation by providers. The article then suggests several ways that any future selection challenges could be addressed, should they occur, without requiring mandatory participation.

The Evidence On Selection Related To Voluntary Participation

As described below, outside observers, papers published by think tanks, academic research, contractors hired by the Innovation Center to provide an independent evaluation of the demonstrations, as well as reports by the Innovation Center itself neither found the existence of selection bias nor recommended mandatory participation. These experts offered many suggestions on how the Innovation Center could achieve its stated objectives, but mandatory participation was not among them.

The ACO model is one of the Innovation Center’s longest running demonstrations, albeit in different forms over time, which attempts to measure cost-effectiveness and quality of treatment. It is the early experience of this model that most proponents of mandatory participation cite as proof of selection bias, primarily because so many providers dropped out at the beginning of the demonstration. Only 123 (36 percent) of the 339 ACOs entering the program between 2012 and 2014 were still participating in 2020.

There were several reasons for the attrition, much of which occurred early in the program. Many of the provider groups were small and ill-equipped to provide the complex reporting and data required by the Innovation Center. The participants did not understand the convoluted requirements before they enrolled, and only thereafter realized they were very unlikely to achieve savings, and therefore the bonuses offered by the Innovation Center. Moreover, participants did not have the processes, experience, or capital to ultimately assume the downside risk required later in the demonstration. In accordance with the rules of the program, they were allowed to drop out and did so.

An outside evaluator concluded: “Pioneer ACO stakeholders also noted that the relationship between the ACOs’ activities and their financial results were not well understood or articulated and that they struggled to firmly understand the Pioneer model rules such as the beneficiary alignment algorithm and financial benchmark calculations…[which] raises the question of whether the alignment algorithm may de-align or not align beneficiaries who are less healthy.”

Other credible sources determined that voluntary participation did not result in adverse selection. For example:

• One analysis concluded: “We also find no evidence that ACOs systematically manipulated provider composition or billing to earn bonuses…. Robustness checks revealed no evidence of residual risk selection…. Careful examination of selection issues revealed that these findings were not driven by nonrandom participation.”
• A study published by the Brookings Schaffer Center concluded: “Evidence suggests that there was minimal systematic patient-level risk selection by ACOs in the first three years of the Medicare Shared Savings Program (MSSP).”  
• An internal CMS evaluation noted: “It does not appear that participants are selecting healthier patients.”
• The Innovation Center engaged outside experts to evaluate the operation of each demonstration, several of whom included explicit conclusions about selection bias. One expert concluded: “This finding suggests that AIM [AIM Investment Model] ACO participant changes over time did not result in selection of certain types of beneficiaries, on average.”

No evaluator of the many demonstrations suggested that that mandatory participation was necessary to produce better results. In one demonstration, the third-party reviewer concluded that the results of the mandatory model were no better than voluntary models. One study directly compared results for mandatory verses voluntary participation and concluded: “spending changes did not differ between the voluntary and mandatory hospitals. This result does not support the concept that organizations perform better when self-selecting into programs.”

If Not Selection, Then What?

While adverse selection did not distort model results, studies did show that there was a myriad of other factors that plagued the initial demonstrations and persisted throughout much of the first decade. A common complaint by providers was a lack of timely data from the Department of Health and Human Services on demonstration operation and performance. One CMS internal evaluator lamented that the inability of CMS technology systems to perform basic tasks for value-based care, including providing performance data to participants, was a key contributor to the reasons providers dropped out.

Additionally, in a 2020 Medicare Payment Advisory Commission meeting, commissioners expressed the view that the multiplicity and overlap of demonstrations made it difficult for participants to sort out the effects of one demonstration from the other. This burden of sorting through the complex requirements for providing data and reports, and inconsistent reporting specifications between the demonstrations, caused many smaller participants to quickly drop out of the demonstrations.

The benchmark calculations, which were intended to measure providers’ effects on costs, were too narrowly drawn and created disincentives that increased over time. The use of historical performance for providers could lock in original calculations of savings/costs. Savings by providers with high-cost patients resulted in lower future benchmarks, which made it more difficult to continue to achieve savings, reducing the incentive to do so.

Many of the shortcomings of previous demonstrations were recognized by the Innovation Center in its assessment of the first decade. The review included a number of suggestions, including health equity a centerpiece of every model; reducing the number, complexity, and redundancy of the many models; re-evaluating how the Innovation Center designs financial incentives to ensure meaningful provider participation (presumably including mandatory participation given the director’s previous comments); better enabling participants to handle down-side risk by providing the tools to participate; reducing the complexity of establishing benchmarks; and expanding the definition of success to include lasting transformation and a broad array of quality investments, rather than focusing on each model’s cost and quality.

Looking Ahead

Despite evidence to the contrary, the Innovation Center has not publicly dropped its position that adverse selection is a problem and that the solution is to require mandatory participation by providers.

Even if selection remains a concern for the Innovation Center, there are ways to detect and correct for selection. One alternative is the expanded use of risk adjusters to assess each participant’s risk before, during, and after the demonstration. Risk adjustment, which is typically used to establish initial payment rates, can also be used to evaluate the risk pools of participants at the end of demonstrations, with payments and shared savings adjusted accordingly.

If benchmarks remain the comparator, risk adjustment will become more important, especially as applied to high- and low-cost beneficiaries as benchmarks converge over time. Risk adjustment is and will continue to be an imperfect process but can be improved by better data, improved statistical techniques, and perhaps, artificial intelligence.

Evidence from many different sources shows that adverse selection has not heretofore been an issue and is not a cause of the failures of past Innovation Center demonstrations in meeting the objectives of savings and quality. Other factors in the operation of the demonstrations are much more likely to explain the results. If selection should ever become an issue, there are ways to adjust models other than forcing providers to participate.