Choice of estimands and estimators affected the interpretation of results for some outcomes in a cluster-randomised trial (RESTORE) due to informative cluster size

Background and objective In cluster-randomised trials (CRTs), different estimands can be targeted, such as the individual- or cluster-average effect. These two estimands can differ in magnitude when outcomes or treatment effects vary with cluster size (termed informative cluster size (ICS)). When ICS is present, commonly used estimators for CRTs, such as mixed-effects model and generalised estimating equations with an exchangeable correlation structure (termed GEEs(exch)), can be biased for both these estimands. With little documented evaluation of ICS, it is currently unknown how commonly it occurs in practice. The aim of this work was to explore whether ICS is present in a published CRT and to investigate its impact on trial results. Methods We re-analysed the RESTORE CRT, which compared protocolised sedation with usual care for critically ill children. For each outcome, we first modelled the association between cluster size and outcome/treatment effect; next, we assessed the impact of ICS by comparing differences between (i) individual- vs. cluster-average estimates and (ii) estimates from mixed-effects models and GEEs(exch) (which can be affected by ICS) to those from IEEs (which are robust to ICS). Results We found evidence of an association between cluster size and either outcomes or treatment effects for 16/33 outcomes (48%). This led to statistically significant differences between the individual- and cluster-average treatment effects for 5 of 33 outcomes (15%). There were >10% differences between (i) individual- and cluster-average treatment effect estimates for 17 outcomes (52%) and (ii) estimates from mixed-effects models/GEEs(exch) and estimates from unweighted IEEs for 13 outcomes (39%). For some outcomes, differences in the choice of estimator or estimand led to differences in the interpretation of results. For example, for the outcome postextubation stridor, the individual-average estimate showed a significant harmful effect (OR=1.65, 95% CI 1.02 to 2.67), unlike the cluster-average (OR=1.38, 95% CI 0.87 to 2.19) or GEEs(exch) estimate (OR=1.57, 95% CI 0.98, 2.50). Discussion ICS can occur in CRTs, and the use of estimators that are not clearly aligned to the target estimand can affect the interpretation of some results.