Vibration of effects resulting from network geometry in mixed-treatment comparisons: a case study on network meta-analyses of antidepressants in major depressive disorder.

ObjectiveIt is frequent to find overlapping network meta-analyses (NMAs) on the same topic with differences in terms of both treatments included and effect estimates. We aimed to evaluate the impact on effect estimates of selecting different treatment combinations (i.e. network geometries) for inclusion in NMAs. DesignMultiverse analysis, covering all possible NMAs on different combinations of treatments. SettingData from a previously published NMA exploring the comparative effectiveness of 22 treatments (21 antidepressants and a placebo) for the treatment of acute major depressive disorder. ParticipantsCipriani et al (2018) explored a dataset of 116 477 patients included in 522 randomized controlled trials. Main outcome measuresFor each possible network geometry, we performed a NMA to estimate comparative effectiveness on treatment response and treatment discontinuation for the treatments included (231 between-treatment comparisons). The distribution of effect estimates of between-treatment comparisons across NMAs was computed, and the direction, magnitude, and statistical significance of the 1st and 99th percentiles were compared. Results4 116 254 different NMAs concerned treatment response. Among possible network geometries, 172/ 231 (74%) pairwise comparisons exhibited opposite effects between the 1st and 99th percentiles, 57/231 (25%) comparisons exhibited statistically significant results in opposite directions, 118 of 231 (51%) comparisons derived non-robust results in terms of statistical significance at 5% risk and 56/231 (24%) treatment pairs obtained robust results across meta-analyses. Comparisons based on indirect evidence only were associated with greater variability in effect estimates. Comparisons with small absolute values observed in the complete NMA more frequently obtained statistically significant results in opposite directions. Similar results were observed for treatment discontinuation. ConclusionIn this case study we observed that the selection of treatments to be included in a NMA could have considerable consequences on treatment effect estimations. Registrationhttps://osf.io/mb5dy