## Tutorial 5: Statistical Tests to Evaluate Bias

Egger's Test

Focusing on funnel plot symmetry, Egger et al. (1997) used a simple but effective regression-based test to detect PB. Specifically, they proposed to regress the standardized effect size (i.e., the effect size divided by its standard error) against precision and then test for nullity of the intercept. In the case of no PB, the intercept of the regression line should be zero, while if PB is present, the intercept deviates from zero due to the asymmetry. There are several variations of Egger’s test in the framework of univariate meta-analysis. Macaskill et al. (2001) introduced a different regression approach to detect PB, where the effect size is regressed on sample size of the study with weights being the inverse of the variances. A similar approach was suggested by Peters et al. (2006), where the sample size is replaced by its inverse.

Begg's Test

Begg and Mazumdar (1994) proposed a nonparametric rank correlation method for PB based on Kendall’s tau. Their method tests whether the standardized effect size and the variance of the effect size are significantly associated. An estimated Kendall’s tau rank correlation coefficient that is close to zero indicates a high degree of independence between the variance and the standardized effect size, suggesting the absence of PB. On the other hand, an estimated Kendall’s tau rank correlation coefficient that is significantly greater than zero indicates the presence of PB. Possible variations of Begg’s rank test can be obtained by replacing standard errors by the inverse of total sample size. However, Begg’s rank test may be underpowered if the number of studies in a meta-analysis is only moderate (Sterne et al., 2000).

Trim and Fill Method

The Trim and Fill method is a popular nonparametric method to detect and correct for PB. The idea behind this method is to add studies to an asymmetric funnel plot until it becomes symmetric. Duval and Tweedie proposed three estimators of the number of missing studies (Duval and Tweedie, 2000). Under the null hypothesis, the approximate distributions of the three estimators were derived and can be used to test for PB.

# Egger's linear regression test

ms1 <- metabin(Ee, Ne, Ec, Nc, data=data1)

summary(ms1)

funnel(ms1)

metabias(ms1, method="linreg")

reg <- lm(I(ms1$TE/ms1$seTE)~I(1/ms1$seTE))

summary(reg)

radial(ms1)

abline(reg)

# Begg's rank-correlation test

ms1 <- metabin(Ee, Ne, Ec, Nc, data=data1)

summary(ms1)

funnel(ms1)

metabias(ms1, method="rank")

# Trim-fill method

ms1 <- metabin(Ee, Ne, Ec, Nc, data=data1)

summary(ms1)

funnel(ms1)

tf1 <- trimfill(ms1)

class(tf1)

funnel(tf1)

print(tf1,digits=2,comb.fixed=TRUE)

Tutorial 6: Multivariate Trim and Fill Method