Health benefit assessment of pharmaceuticals: An international comparison of decisions from Germany, England, Scotland and Australia

- We identified substantial disagreement between the FJC and NICE, SMC and PBAC.
- FJC and each agency agreed in 40%, 47.6% and 48.7% of ratings.
- Agreement improved moderately when comparing decisions on effectiveness only.
- Generally, the FJC tends to appraise stricter than NICE.

BackgroundLittle is known on the performance of the newly introduced health benefit assessment process, AMNOG, in Germany compared to other health technology assessment agencies.ObjectiveWe analysed whether decisions of the German Federal Joint Committee (FJC) deviate from decisions of the UK National Institute for Health and Care Excellence (NICE), the Scottish Medicines Consortium (SMC) and the Australian Pharmaceutical Benefits Advisory Committee (PBAC).MethodsWe analysed decisions made for comparable patient subgroups by the four agencies between 2011 and 2014. First, decisions were compared (a) by their final outcome, i.e. whether a health benefit was identified, and (b) by the agencies' judgement on comparative effectiveness. Subsequently, we partially explored reasons for differences between HTA agencies.ResultsFrom the 192 FJC decisions, we identified 55 that overlapped with NICE, 166 with SMC and 119 with PBAC. FJC agreed with NICE in 40% in final outcome (Cohen's Kappa = −0.13). Similar results were obtained for FJC and SMC (47.6%, kappa = 0.03) and FJC and PBAC (48.7%, kappa = 0.07). Agreement increased when comparing judgements based on comparative effectiveness only. However, the FJC's final decision was positive only in 43.6%, 39.2% and 44.5% of the patient subgroups, as opposed to 74.5% (NICE), 68.7% (SMC), and 68.9% (PBAC), respectively.ConclusionWe show that the FJC - an agency relatively new in structurally assessing the health benefit of pharmaceuticals - deviates considerably in decisions compared to other HTA agencies. Our study also reveals that the FJC tends to appraise stricter than NICE.