Challenges facing the use of economic evaluations in the Austrian system
When contrasting the Austrian system and the three case studies with the general challenges for using economic evaluation in the previous section (“Challenges to the application of economic evaluations”), the biggest limitation seems to be inherent in the Austrian health care system characteristics. Firstly, the Bismarck system means that—although health care resources are of course not infinite in Austria—the available budget is not as fixed in advance as it is the case in a Beveridge country such as the UK. However, as other cases with Bismarck-based systems demonstrate (Belgium, the Netherlands), a health insurance-based funding in itself does not seem to be the key challenge. It seems that rather the combination with other system characteristics that are inherent in the Austrian system such as fragmentation of funding, governance, and service provision play an important role. Not least, lack of transparency in the decision-making culture has been described as being a core barrier to the inclusion of (cost)-effectiveness evidence into the decision-making processes [
24,
36]. For example, resource allocation decisions are often passed on to the level of the clinicians rather than addressing them at the macro-political level (especially in the inpatient sector [
37]), which prevents the use of economic evaluation. Several initiatives have now started at the hospital-provider level to systematically evaluate high-cost hospital drugs before funding approval; however, CEA and CUA have so far not systematically integrated into the evaluation concepts [
38]. In the case of outpatient drugs, where a form of positive list exists, according to the Austrian law, drugs that are excluded from the list can still be publicly reimbursed if approved by the chief medical officer (based on the individual needs of a patient). Thus, both the culture and the legal basis that form the Austrian health care system hinder a rational approach to resource allocation and thereby the use of economic evaluation. This does not mean that rationing is absent, but it rather indicates that rationing, which de facto always exists in a situation where demand exceeds supply, occurs implicitly.
The case studies demonstrate that efficiency generally has a low priority compared to other reimbursement criteria. As shown in the case of HPV vaccination, the ICER did not play a role in the discussions, while the burden of disease (cervical cancer incidence and mortality in Austria), the long-term impact on cancer epidemiology, safety issues, and the budget impact were all subjects of the political discourse [
39,
40]. Even in the absence of a threshold, cost-effectiveness studies could be used to identify preferable patient subgroups. The final decision to publicly pay for vaccinating boys and girls despite a very high ICER demonstrates that this type of evidence is of low priority in justifying decisions. The case rather indicates that the health benefit, affordability, and prices are the core criteria for decision makers, rather than cost-effectiveness. This is also confirmed by the TAVI case, as it has been shown that decisions are based rather on clinical parameters alone, such as severity of aortic stenosis, age, surgical risk, life expectancy, and comorbidities, than on cost-effectiveness results. Efficiency seems at best to be used retrospectively to justify resource allocation decisions that were originally made based on affordability grounds or based on limited capacities that were required to restrict access to a manageable number of patients.
Other challenges outlined in “Challenges to the application of economic evaluations” also seem to be confirmed. Regarding applicability, data quality, and reporting/communication, members of the drug evaluation committee pointed out early that reasons for rejecting the use of economic evaluations were that the presentation of results was unsatisfying and difficult to understand. Furthermore, studies lacked transparency and the application of them to the decision problem was very limited because an inappropriate comparator was used or the study was not based on relevant Austrian data [
28]. Some of these deficiencies still seem to exist. As the diabetes case study demonstrates, one reason for neglecting the study may have been its lack of transparency. Furthermore, the outcome parameter (€ per % of HbA1c reduction) and the time horizon (1 year despite the fact that diabetes is a chronic disease) used may likely have been meaningless for the decision problem, all of which resulted in low relevance for the Austrian decision context and low credibility of the study in general. Mayer et al. [
17] showed that there are particular quality problems with cost data in Austria because of inconsistencies in the costing methods and cost-reporting standards. This poses an additional limitation to using reviews of economic evaluations (such as the TAVI case presented earlier), because it restricts the generally limited transferability of international study results even further.
Furthermore, the lack of methodological expertise and knowledge seems to be relevant in Austria. Members of the drug evaluation committee expressed concerns regarding their expertise in economic evaluations in a survey in 2006 [
28]. Although more than 12 years have passed since this survey, none of the members of the drug evaluation committee is a (health) economist by training. The lack of knowledge is also visible in other decision processes, as an interview among members of the Austrian vaccination committee demonstrated [
39]. The interviewee claimed that the HPV study demonstrated that the vaccination is not cost effective. However, this was mentioned nowhere in the report. In fact, in the absence of a cost-effectiveness threshold, a judgement on cost effectiveness based on the ICER is not even possible and has made interpretation of studies difficult (also demonstrated in the TAVI case). It seems to be common that economic terms are often used without a clear understanding of their meaning, which confirms the international findings outlined in “Challenges to the application of economic evaluations.”
A potential way forward
The analysis has demonstrated that the challenges with using economic evaluations that have been identified in the international literature seem to also exist in Austria. It is therefore not surprising that economic evaluations have played a negligible role in Austrian coverage processes so far. Some of the challenges could be overcome quite easily if there is agreement among decision makers to systematically use economic evaluations as a source of evidence in reimbursement processes. For instance, education and capacity building will reduce the lack of knowledge among decision makers and increase the capacities of skilled staff involved in critical appraisal of industry studies, which, in turn, will reduce wrong perceptions and misunderstandings. Furthermore, standardization of methods and presentation of results as well as detailed quality criteria will increase the methodological quality of studies, their relevance for the decision problems, and the trust in the studies. To increase transparency, manufacturers could be obliged to submit not just the final study, but also the calculations the study is based on, as is the case in other countries (e. g., UK, the Netherlands, Finland, Sweden), where the decision analytic model has to be submitted alongside the study report. The methodological quality could be increased by a mandatory detailed guideline that clearly sets out the methodological standards and the reporting requirements for industry submissions, but also for economic studies that are commissioned to independent bodies. This shift towards more transparency of the studies would also reduce the perceived risks that economic evaluations may be misused. The submitted models may then be used by the decision makers themselves for analyzing different scenarios (e. g., different price scenarios).
However, the biggest barrier to integrating economic evaluations in reimbursement processes seems to lie within the nature of the health care system. On the one hand, the nature of the studies and the theoretical and ethical principles they are based on are in part incompatible with the Austrian legal context, the system characteristics, the moral values entrenched in the system, and the political culture. As a consequence, some of the currently existing methodological requirements for economic evaluation in Austria (e. g., to apply a health care system perspective and ignore out-of-pocket costs [
25]) are in fact debatable within a Bismarck type of health care system. On the other hand, further neglecting of the notion of efficiency will threaten long-term sustainability of the health care system.
Copying how economic evaluations have been integrated into decision making elsewhere will likely fail, even if they refer to the same health care system type such as the Netherlands or Belgium, which are also rooted in the Bismarckian tradition [
23,
41]. The situation rather calls for a process that takes the system characteristics into account. Increasing the use of economic evidence first of all requires an understanding among decision makers at all levels that addressing efficiency as one of a number of other decision criteria (and the rationalization and rationing issues that this brings with it) is not unethical, but that it may be rather unethical to ignore efficiency [
23]. By passing on resource allocation decisions to clinicians, it remains nontransparent what type of criteria are applied and whether they reflect societal values (e. g., postcode rationing, allocation of resources based on health literacy, etc.). The value of CEA or CUA for decision makers in that sense is that it contributes to accountability by reassuring payers that their money is being spent wisely and by reassuring patients, caregivers, and the general public that their contributions to the health service are treated consistently [
3]. The premise is, however, awareness that by using CEA or CUA, some discourse about willingness-to-pay based on a utilitarian philosophy would have to be introduced and that maximization of health would have to become one explicit out of several further decision-making criteria.
To tackle the applicability issue, a process needs to be initiated aiming at identifying what the objectives and needs of the decision makers precisely are [
18,
22], i. e., what type of economic evidence they would find helpful and in what way this should be presented to facilitate its use. Both decision makers from various parts of the system and experts in economic evaluation and other forms of efficiency analysis would need to take part in this process. Existing reviews from other countries’ guidelines [
42] and suggestions that have already been developed for Austria based on international examples [
43] can serve as a starting point. But as the German case has shown [
44], the process will also need to facilitate methodological discussions and it will need to be open towards adaptive and new forms of economic evaluations that may differ from the methods that are currently applied in other jurisdictions.
A starting point may be to draw on the suggestions from Williams and Bryan (2007) [
11] that are to shift away from normative to more positive economics. Rather than a CEA or CUA, the primary type of economic evaluation to be used would then be a cost-consequence analysis that displays different effects and costs in a disaggregated way without combining results into a single indicator (such as an ICER). Instead of defining cost effectiveness based on willingness-to-pay thresholds in a normative (prescriptive) way, such studies describe the likely consequences in terms of costs and health which may be more approachable by decision makers [
45,
46]. However, this needs to be weighed against the disadvantages of cost-consequence analysis: disaggregation results in limited comparability and generalizability; additionally, the absence of a single cost-effectiveness ratio may lead to less consistent decision making. Finally, leaving the weighing of the relative importance of different costs and benefits to decision makers bears the risk of cherry picking [
45,
46]. Nevertheless, presenting costs and a range of outcomes in disaggregated form increases transparency, mitigates uncertainty, or makes more clearly visible what the core uncertainties are. Thereby, decision makers are supported with additional knowledge to be used to make informed decisions and to justify the decisions later on, including the value judgement they are based on. As the HPV case study has shown, the disaggregated presentation of knowledge may serve other purposes than just supporting efficient resource allocation (e. g., providing information on equity and affordability, requirements for program implementation, long-term health gains to be expected, etc.). Hence, it reduces the asymmetry of information that usually exists between payers and providers of interventions [
47].
Regarding the process of conducting individual studies, the studies will likely be more relevant if decision makers and those who conduct the studies engage more actively in a communication process before the research starts, whereby the parameters to be addressed in the economic study are clarified (e. g., the type of comparator or the outcome measure used).
In parallel to discussions on efficiency and economic evaluation, a discussion on further attributes of benefit that may be valued alongside health gains (e. g., severity of a disease) needs to be initiated and criteria to represent those attributes need to be defined and made transparent.