Bayesian meta-analysis of studies with rare events: Do the choice of prior distributions and the exclusion of studies without events in both arms matter?

1 Title 1 Bayesian meta-analysis of studies with rare events: Do the choice of prior distributions and 2 the exclusion of studies without events in both arms matter? 3 4 Authors: 5 Soheila Aghlmandi, Peter Jüni, James Carpenter, Marcel Zwahlen 6 1 Institute of Social and Preventive Medicine (ISPM), University of Bern, Mittelstrasse 43, 7 CH-3012 Bern, Switzerland 8 2 Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, 9 Spitalstrasse 12, CHBasel, Switzerland 10 3 Applied Health Research Centre (AHRC), St. Michael’s Hospital, Department of Medicine, 11 University of Toronto, 30 Bond Street, Toronto, Ontario M5B 1W8 12 13 4 London School of Hygiene and Tropical Medicine (LSHTM), University of London, Keppel 14 Street, London WC1E 7HT 15 16 17 Corresponding author: 18 Soheila Aghlmandi 19 soheila.aghlmandi@usb.ch 20 21 22 Short title: Bayesian meta-analysis of rare events 23 Word counts: 4700 24 Number of figures: 5 25 Number of tables: 6 26 Number of references: 30 27

excluding studies with no events in both arms for meta-analyses introduced bias into the 66 pooled estimates when there was no true treatment effect. 67 Another approach uses a continuity correction (CC) of 0.5 for each cell [6,7]. Sweeting et al. 68 [8] have proposed different CCs that perform better if the number of patients in the treatment 69 and control groups are severely imbalanced. Based on simulation studies, [4] suggests that 70 deleting trials with no events in either arm or adding CCs can introduce bias to the calculation 71 of effect measure(s).

72
Various statistical methods have been proposed for using and combining information from 73 trials with no events. A principled approach is to assume that the number of events given n 74 (the number of patients in a treatment group) and the true risk follows a binomial distribution. 75 Kuss [4] used beta-binomial regression methods to make inferences about OR, RR, and risk 76 difference. Kuss's approach assumes that events in the treatment groups are binomially 77 distributed, i.e. the likelihood for the observed events is the binomial distribution, and it can 78 handle studies with no events. Cai et al. [9], proposed a method that uses the idea of 79 conjugacy in the same way as the beta-binomial model. They used Poisson models for both 80 fixed effect (FE) and random effects (RE) MA to make inferences about the RR between two 81 treatment groups. Bohning et al. [10] proposed a Poisson model for RE and concluded that 82 these techniques returned almost the same results as the Mantel-Haenszel (MH) method. 83 Other methods along these lines can be found in serveral other publications [11][12][13][14][15][16][17][18][19]. 84 Another approach to the MA of rare events is to take a fully probabilistic, Bayesian approach. 85 Here, after the specification of prior distributions for all relevant parameters of the analysis 86 model, the data and application of Bayes's theorem allows obtaining posterior distributions 87 for all relevant parameters [20]. Smith (1995) and Warn [21,22] showed how to implement a 88 fully Bayesian FE and RE meta-analysis with exact binomial likelihood using WinBUGS. 89 This of course needs a decision about the prior distributions to be used that could reflect 90 expert opinion or be derived from external available information [23], or that could be set to 91 reflect vague prior information. In an MA of rare events, the data contain limited information, and the information of the prior distributions is expected to contribute to the posterior 93 distribution. Sweeting et al. [8] investigated, among other approaches, Bayesian inference in 94 the FE meta-analysis in situations with rare events, and concluded that the method provided 95 good coverage in all scenarios investigated. However, they excluded a priori trials with no 96 events in both arms from the MA. 97 We used a Bayesian approach to conduct the MA of studies with rare events to estimate the 98 odds ratio, more precisely the log of the odds ratio, and specifically assessed the importance 99 of (1) excluding yes or no trials with zero events in both arms, and (2) the choice of priors for 100 the true OR and τ for the heterogeneity in case of RE meta-analyses. We chose the OR as the 101 target effect measure for ease of implementation because it is almost identical to the risk ratio 102 in rare event situations and allows easier model implementation using the logit function. In 103 Section 2, we define the statistical model and the different types of priors to be used both in 104 FE and RE meta-analyses. In Section 3, we describe a simulation study and the range of 105 scenarios in which we varied assumptions about true OR, the heterogeneity τ in RE standard 106 deviation, the risk in the control group, the total number of patients in treatment and control 107 groups, and the randomization ratio in the studies. In Section 4, we present the results of the 108 simulation studies. In Section 5, we reanalyze studies on the cardiovascular risk of 109 Rosiglitazone in the treatment of Type II diabetes. 2 Bayesian approach to meta-analysis of studies with rare events 112 Two approaches can be used to combine study findings: 113 1) The FE MA assumes that the treatment effect is the same in all of the studies. For FE, we 114 consider that observed variation is caused by sampling variation.

115
2) The RE MA assumes that there is a variation of the true treatment effect across studies 116 (heterogeneity). Therefore, one makes additional assumptions on how the study-specific 117 treatment effects vary. In the binary case, one commonly assumes that the study-specific 118 log(ORi) follow a normal distribution, which then implies that one also estimates the standard 119 deviation τ of this normal distribution [24]. No less than 16 methods have been identified to 120 estimate τ or τ-squared [25]. In situations with rare events, it is particularly challenging to 121 estimate τ and the choice of the prior distributions for τ is expected to be important. 122 2.1 Model structure for the meta-analysis 124 Throughout, we assume that data for each individual study � = (1, … , �) in the MA come 125 from a two-arm randomized trial comparing a new treatment (received by the treatment group 126 t) with a control treatment (received by the control group c) and that the outcome assessed in 127 the MA is a binary adverse event. The numbers of events for c and t groups in each study � 128 then follow a binomial distribution all studies to be meta-analyzed, i.e. ���(�� � ) = ���(��).

137
For the Bayesian approach, prior distributions must be specified for all unknown parameters,  scale has been proposed and used in previous studies [8,21,22,27]. Therefore, we used a normal distribution with a mean of zero and SDs of 10 and 100 (precisions of 0.01 and 153 0.0001). To cover very small baseline risks, it seems reasonable to use these values for SDs. 154 We also used uniform distribution with range of 20, which, when back transformed to the 155 risk scale, has a substantial mass close to zero, but is bounded away from zero at 2 × 10 -/ . standard deviation (9). In addition to this structural assumption, one needs to specify 162 prior distributions for both the mean (8) and the standard deviation (9). To reflect a rare 163 events situation, we chose a uniform distribution U(-6 to -3) for 8 and U(0 to 1) for 9.

164
These specifications provide a 95% prior interval of 0.16% to 7.0% for the risk in the 165 control group.
. 172 We specified a ��7���(�1�� = 0, @A = 10) distribution as the prior distribution for 173 log(��) and investigated several prior distributions for ? as given in Table 2. Because it is particularly challenging to estimate τ in situations with rare events, we expected the 175 specification of the prior distributions for τ to be important. Working on the log(�� � ) implies 176 Table 2. List of prior distributions for ?
that a ? of 0.5 to 1.0 already reflects large heterogeneity of the treatment effects across 177 studies, as discussed in Spiegelhalter [26]. Therefore, we set two prior distributions to have a 178 mean of 0.5,and a third, the uniform(0, 2), had a mean of 1. Finally, we used one of the prior   (Table 3). 208 � Sample size of a single study: We also used a uniform distribution to simulate the 209 sample size of each study.   (2). For different ratios, when we increased the proportion of zeros in both arms, bias increased slightly in a negative direction, but coverage was roughly the same.

254
The uniform and normal distribution with SD of 100 behaved similarly with respect to 255 both coverage and bias. Normal distribution with smaller SD (10)  increasing the information on the control group, the coverage dropped to 93% and the 276 bias increased in the negative direction. For τ = 0.5, the observed coverage was lower 277 for 1:1 randomization than τ = 0.2 but similar to the other randomization scenarios.

278
In summary, uniform distribution is a poor choice to account for heterogeneity in RE

279
MA due to high bias from true log(OR). The mean coverage for log(OR) was similar for all the specified priors for � �� , but 283 different for scenarios with higher true heterogeneity τ = 0.5, on average 93.5% and 284 85%, respectively. Bias was smaller for τ = 0.2 than τ = 0.5 for both true log(OR).

285
Both mean coverage and bias were similar for low or high proportions of zeros in 286 both arms irrespective of true log(OR). For different randomization scenarios (1:1, For all the scenarios with small to moderate heterogeneity for both true log(OR)s, 295 coverage returned by the Bayesian methods was above 94% and there wass no 296 specific pattern of increase or decrease when we had imbalanced randomization. In 297 contrast, bias increased towards the negative by putting more information in the 298 control group. The coverage was lower, 93% on average, for high heterogeneity (0.5) 299 and the estimates were biased for true log(OR) with no specific direction. There was a 300 clear pattern of increase in the coverage when we had more than 30% zeros in both 301 arms for 0.5 heterogeneity scenarios ( Table 5 and Table 6).

305
In general, for all the RE Bayesian methods in the different data scenarios, the average 306 coverage and bias were almost identical whether studies with no events were included or 307 excluded. Bayesian methods provide good coverage of 94% on average, slightly higher than 308 coverage when using the MH method, 92.6%, but both methods have a slight bias of the point 309 estimate for the true log(OR). For log(1), null effect, bias was surprisingly large, especially 310 for the scenarios in which there was high heterogeneity (0.5). By increasing the information 311 in the control group, we observed an increase in bias, but coverage remained similar. As the 312 proportion of zeros in the data increased, the hierarchical model with half-normal prior for ? 313 showed better coverage and gave a less biased estimate compared to using a uniform 314 distribution for ?. Estimates from the MH method displayed evidence of bias and poor 315 coverage because the method was unable to account for heterogeneity when the standard 316 deviation in the RE data generation scenario was high (0.5). We assigned treatment vs. control group for the ratio of group sizes b deletion is a logical argument; zero means trials with zero in both arms are excluded from the analyses. c The Gelman and Rubin diagnostic is used to check the convergence of multiple mcmc chains run in parallel. d Percentage of trials with no events in both arms.  The Bayesian methods are illustrated with data from a meta-analysis of 48 comparative trials 333 that examine the possible cardiac toxicity of Rosiglitazone in RCTs designed to study 334 cardiovascular morbidity and mortality. Rosiglitazone, a Type II diabetes medicine, was 335 introduced in 1999 and is known to reduce blood glucose and glycated hemoglobin levels.

336
Adverse events of Rosiglitazone were studied and categorized as rare events. We used the 337 MA data, which [27] also used. Events are rare for myocardial infarction (MI): 26 trials had 338 zero in one arm, 10 trials had zero in both arms. The rare events problem is more pronounced 339 for cardiovascular (CV) death since 25 studies had no events in both arms, and 17 had one 340 arm with no event (the full data set is in supplemental Table S1). We illustrated the situation 341 with this example using a selection of our Bayesian methods, and compared the results to the 342 MH and Peto methods. We also compared our results with those reported by [11], and logistic 343 regression (LR) by [27]. In FE, when we used a normal distribution with SD of 100 for the prior distribution of the 349 logit of � �� , the estimated OR was higher (OR = 1.43) than in all the other Bayesian 350 approaches, and results were in line with both the MH and Peto methods (OR = 1.429 and 351 1.430) and logistic regression applied by [27]. For RE Bayesian, with the same prior for 352 the logit of � �� and a half-normal distribution (mean = 0.5) for the prior distribution of ?, 353 we observed an OR of 1.45, which also was higher than the estimates from the other  Shuster's RE model estimation is higher than our estimations with wider confidence 358 interval than our CIs.  � In FE, Bayesian approaches' highest OR was 1.62, which is estimated by norm(0, 100) on 366 logit of � �� , and the 95% CI is slightly wider than other priors on baseline risk. We 367 observed the same results for RE Bayesian approaches with the same prior on the risk of 368 control group with half-normal (mean = 0.5) as ?, but the CI is even wider for RE than for 369 FE. also the widest 95% confidence interval.

377
The high sensitivity to the choice of priors in CV death of Bayesian methods can be 378 explained due to very low event rate, 0.5%, while for MI it is almost 2%. proposed and the results obtained seem to depend on the approach chosen [4,8,10,14,18,384 30]. In addition some computational difficulties might occur, especially if one attempts to use 385 a random-effects model because the available information is low when analyzing rare events.

386
Here we focused on assessing the variability of the results, in terms of bias and coverage, for 387 Bayesian approaches to implementing the MA. The fully probabilistic (Bayesian) analysis via 388 MCMC methods has the advantage that exact binomial likelihoods can be used, and that 389 studies with zero events in both arms do not need to be excluded from the analysis. However,

402
For the simulated data scenarios with varying true log(OR) across the studies in the MA, the 403 results of the Bayesian meta-analyses were also sensitive to the specification of the prior 404 distributions for heterogeneity parameter ?. We found that using a uniform prior distribution 405 from 0 to 2 resulted in high bias and lower coverage. Also, using lognormal distribution 406 suggested by Turner et al. [28] for ? D resulted in slightly better results compared to uniform 407 distribution but, using an informative prior exemplified by half-normal with mean = 0.5 for ? 408 performed better.

409
In summary, in Bayesian MA of rare events the bias for the point estimate for the log(OR) 410 and the coverage of the Bayesian CIs were similar whether studies with no events in both 411 arms were excluded or not. However, bias and coverage were sensitive to the specification of 412 the prior distributions for risk in the baseline groups and for the between-study heterogeneity.