Hospinnomics’ 2020 technical workshop on risk adjustment: a synthesis
The single-payer model and the adoption in 2004 of an activity-based payment scheme for hospitals have kept France away from risk-adjustment for a long time until the 2018 Social Security Financing Law allowed the development of experimental payment schemes. Since then, three experiments using risk-adjusted payments have been developed. One experiment (EDS) aims at providing an episode-based bundled payment for three procedures: total hip replacement (THR), total knee replacement (TKR), and Cancer colectomy (CC). The two other experiments (PEPS and IPEP) aim at remunerating groups of health care professionals based on both their practice population characteristics and their patients’ outcomes. Those three experiments led to the production of risk-adjustment models by French health administrations.
In this context, Hospinnomics, the University of Montpellier (Grégoire Mercier), and the school of public health (Nicolas Sirven, EHESP) have launched a technical workshop on risk adjustment opened to health economists, statisticians, public health professionals, health care professionals and members of health administrations. Four workshops took place from January 2020 to June 2020, giving those who work on risk-adjustment an opportunity meet and share their questions and progress. The presentations and discussions provided a unique occasion to document the state of the art on risk-adjustment payments in France and to raise the issues associated with their implementation.
Six presentations were given from researchers or representatives of French administrations. The first was on national health care expenditure modeling, presented by Dr. Panayotis Constantinou whose economics Ph.D. thesis focuses on this topic. This first presentation brought out the main difficulties faced when building a prediction model for healthcare expenditure. Nathalie Rigollot, a statistician from the technical agency for information on hospitalization (ATIH), gave the second presentation. This presentation addressed the issue of pathway prediction for TNR, TKR, and CC to define individual risk-adjusted payment for EDS and of individual outcomes adjustment for the IPEP experimentation. The third presentation, by David Bernstein (DGOS, Ministry of Health), was a synthesis of the foreign experimentations using risk adjustment. The fourth one, by Marc-Antoine Sanchez, a PhD student at Hospinnomics and Créteil, was on the introduction of quality indicators in France and the necessity to adjust them on patients’ expected outcomes. The fifth one, by Alexandre Vimont, a Ph.D. student at URC ECO (AP-HP), was on the use of artificial intelligence-based models for healthcare expenditure prediction. The last one was on the measure of efficiency gains in IPEP experimentations to compute a bonus for groups of primary care professionals. It was presented by Victor Bret and Cécile Billionnet from the National Health Insurance Fund (CNAM).
The main issues outlined in the presentations and the following discussions are summarized below.
Model design considerations were addressed in almost all presentations. Different models were introduced and compared for risk prediction: linear regression, general linear model, and nonlinear models. Linear regression has the advantage of being easily understood by health care professionals. GLM models usually outperform linear regression for health care expenditure prediction. The weakness of the GLM models is the multiplicativity of the factors that lead to overprediction for patients with several comorbidities. Nonlinear models as random forests and neuronal networks are better at predicting healthcare expenditure if rich individual data are available. Yet, they make the justification of the mechanism behind resource allocation more difficult to present.
The evaluation of the models’ performance has been at the heart of all discussions. Presenters usually evaluated their model on their predictive power, using R², Root Mean Square Error and Hit ratio. On validation datasets, the gap between predicted and actual spending was also measured to ensure the fairness of the models. The operational impact of risk adjustment models on quality of care and overall well-being has also been raised, but at this stage without precise answers.
The selection of independent variables was also discussed. Differences in health care expenditure can reflect over- or under-use due to supplier-induced demand or inequities in access to care. Therefore, variables must be chosen based on their predictive power but also on the fairness of their expected impact. The ATIH has carried out tests showing that deprivation may lead to an under-use of health care services, subsequently leading to an underestimation of health spending for certain individuals. In those cases, variables have been excluded from the model to keep those that were closer to the true patient health status. On the other hand, the cost of caring for poorer individuals could be higher, and extra costs have to be accounted for in the model. More generally, endogeneity issues have had to be tackled.
Almost all presentations addressed the issue of data completeness. Prediction models use administrative data from the CNAM that only include individuals with a positive medical care consumption for a given year. This is one of the main difficulties faced by researchers when predicting health care expenditure for the general population. Another difficulty is the fact that data capture health care refunding instead of health care status. To address this second issue, the CNAM has created a “map of diseases” that helps identify diseases based on information on health care consumptions. Almost all presenters indicated having used this tool.
One of the special features of the 2018 French experiments on risk-adjusted payments is the desire to co-construct the payment model with health care professionals. Meetings are periodically organized with the Ministry of health, the CNAM, and the experiment leaders to discuss the design of the payment model. While this increases adherence of professionals to the program, it may also lead to strategic behaviors from their part. The rationale given for the co-construction of the model is that professionals have intrinsic motivations to improve care quality which will translate in the choice of the best model for care improvement. Gaming issues have been raised during the workshops, especially for hip and knee replacement pathways, because an explicit threshold is defined above which patients will need to use rehabilitation care, influencing surgeons’ practice.
These are the main subjects addressed during the 2020 Hospinnomics technical workshop and further updates are currently made in order to produce a synthesis on risk adjustment in France. Indeed, many features of the French healthcare system make this experience atypical and a publication would be of interest to other countries.
A final session is organized on 14 January 2021 (5pm-7pm). For this session, Randall Ellis (Boston University) will react to the work currently developed in France and give a presentation on Advanced Risk Adjusters and Predictive Formulas for Diagnosis-Based Risk Adjustment.
To participate in the technical workshop and receive the Zoom link, please register by sending an email to seminaire.hospinnomics@gmail.com.