`anova.rma.Rd`

For two (nested) models of class `"rma.uni"`

or `"rma.mv"`

, the function provides a full versus reduced model comparison in terms of model fit statistics and a likelihood ratio test. When a single model is specified, a Wald-type test of one or more model coefficients or linear combinations thereof is carried out.

# S3 method for rma anova(object, object2, btt, X, att, Z, digits, ...)

object | an object of class |
---|---|

object2 | an (optional) object of class |

btt | optional vector of indices to specify which coefficients should be included in the Wald-type test. Can also be a string to |

X | optional numeric vector or matrix to specify one or more linear combinations of the coefficients in the model that should be tested. See ‘Details’. |

att | optional vector of indices to specify which scale coefficients should be included in the Wald-type test. Can also be a string to |

Z | optional numeric vector or matrix to specify one or more linear combinations of the scale coefficients in the model that should be tested. See ‘Details’. Only relevant for location-scale models (see |

digits | integer to specify the number of decimal places to which the printed results should be rounded. If unspecified, the default is to take the value from the object. |

... | other arguments. |

When a single model is specified, the function provides Wald-type tests of one or more model coefficients or linear combinations thereof.

In particular, for a fixed- or random-effects model (i.e., a model without moderators), this is just the test of the single coefficient of the model. For models including moderators, an omnibus test of all model coefficients is conducted that excludes the intercept (the first coefficient) if it is included in the model. If no intercept is included in the model, then the omnibus test includes all coefficients in the model including the first.

Alternatively, one can manually specify the indices of the coefficients to test via the `btt`

argument. For example, with `btt=c(3,4)`

, only the third and fourth coefficient from the model would be included in the test (if an intercept is included in the model, then it corresponds to the first coefficient in the model). Instead of specifying the coefficient numbers, one can specify a string for `btt`

. In that case, `grep`

will be used to search for all coefficient names that match the string. Using the `btt`

argument, one can for example select all coefficients corresponding to a particular factor to test if the factor as a whole is significant. See ‘Examples’.

Instead, one can use the `X`

argument to specify a linear combination of the coefficients in the model that should be tested (i.e., whether the linear combination is significantly different from zero). If a matrix of linear combinations is specified, each row defines a particular linear combination to be tested. If the matrix is of full rank, an omnibus Wald-type test of all linear combinations is also provided. Linear combinations can also be obtained with the `predict.rma`

function, which will provide corresponding confidence intervals.

When specifying two models for comparison, the function provides a likelihood ratio test (LRT) comparing the two models. The two models must be based on the same set of data, must be of the same class, and should be nested for the LRT to make sense. Note that LRTs are not meaningful when using REML estimation and the two models differ in their fixed effects. Also, the theory underlying LRTs is only really applicable when comparing models that were fitted with ML/REML estimation.

For location-scale models fitted with the `rma.uni`

function, one can use `att`

to specify the indices of the scale coefficients to test. Similarly, one can use the `Z`

argument to specify one or multiple linear combinations of the scale coefficients in the model that should be tested.

An object of class `"anova.rma"`

. When a single model is specified (without any further arguments or together with the `btt`

or `att`

argument), the object is a list containing the following components:

test statistic of the Wald-type test of the model coefficients.

corresponding degrees of freedom.

corresponding p-value.

indices of the coefficients tested by the Wald-type test.

number of outcomes included in the analysis.

number of coefficients in the model (including the intercept).

number of coefficients included in the Wald-type test.

some additional elements/values.

test statistic of the omnibus Wald-type test of all linear combinations.

corresponding degrees of freedom.

corresponding p-value.

description of the linear combinations tested.

values of the linear combinations.

standard errors of the linear combinations.

test statistics of the linear combinations.

corresponding p-values.

log-likelihood, deviance, AIC, BIC, and AICc for the full model.

log-likelihood, deviance, AIC, BIC, and AICc for the reduced model.

number of parameters in the full model.

number of parameters in the reduced model.

likelihood ratio test statistic.

corresponding p-value.

test statistic of the test for (residual) heterogeneity from the full model.

test statistic of the test for (residual) heterogeneity from the reduced model.

estimated \(\tau^2\) value from the full model. `NA`

for `"rma.mv"`

objects.

estimated \(\tau^2\) value from the reduced model. `NA`

for `"rma.mv"`

objects.

amount (in percent) of the heterogeneity in the reduced model that is accounted for in the full model (`NA`

for fixed-effects models or for `"rma.mv"`

objects). This can be regarded as a pseudo \(R^2\) statistic (Raudenbush, 2009). Note that the value may not be very accurate unless \(k\) is large (Lopez-Lopez et al., 2014).

some additional elements/values.

The function can also be used to conduct a likelihood ratio test (LRT) for the amount of (residual) heterogeneity in random- and mixed-effects models. The full model should then be fitted with either `method="ML"`

or `method="REML"`

and the reduced model with `method="FE"`

. The p-value for the test is based on a chi-square distribution with 1 degree of freedom, but actually needs to be adjusted for the fact that the parameter (i.e., \(\tau^2\)) falls on the boundary of the parameter space under the null hypothesis (see Viechtbauer, 2007, for more details).

LRTs for variance components in more complex models (as fitted with the `rma.mv`

function) can also be conducted in this manner (see ‘Examples’).

Wolfgang Viechtbauer wvb@metafor-project.org https://www.metafor-project.org

Hardy, R. J., & Thompson, S. G. (1996). A likelihood approach to meta-analysis with random effects. *Statistics in Medicine*, **15**(6), 619--629. https://doi.org/10.1002/(sici)1097-0258(19960330)15:6%3C619::aid-sim188%3E3.0.co;2-a

Huizenga, H. M., Visser, I., & Dolan, C. V. (2011). Testing overall and moderator effects in random effects meta-regression. *British Journal of Mathematical and Statistical Psychology*, **64**(1), 1--19. https://doi.org/10.1348/000711010X522687

López-López, J. A., Marín-Martínez, F., Sánchez-Meca, J., Van den Noortgate, W., & Viechtbauer, W. (2014). Estimation of the predictive power of the model in mixed-effects meta-regression: A simulation study. *British Journal of Mathematical and Statistical Psychology*, **67**(1), 30--48. https://doi.org/10.1111/bmsp.12002

Raudenbush, S. W. (2009). Analyzing effect sizes: Random effects models. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), *The handbook of research synthesis and meta-analysis* (2nd ed., pp. 295--315). New York: Russell Sage Foundation.

Viechtbauer, W. (2007). Hypothesis tests for population heterogeneity in meta-analysis. *British Journal of Mathematical and Statistical Psychology*, **60**(1), 29--60. https://doi.org/10.1348/000711005X64042

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. *Journal of Statistical Software*, **36**(3), 1--48. https://doi.org/10.18637/jss.v036.i03

### calculate log risk ratios and corresponding sampling variances dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg) ### fit random-effects model res1 <- rma(yi, vi, data=dat, method="ML") ### fit mixed-effects model with two moderators (absolute latitude and publication year) res2 <- rma(yi, vi, mods = ~ ablat + year, data=dat, method="ML") ### Wald-type test of the two moderators anova(res2)#> #> Test of Moderators (coefficients 2:3): #> QM(df = 2) = 33.9192, p-val < .0001 #>#> #> Hypotheses: #> 1: ablat = 0 #> 2: year = 0 #> #> Results: #> estimate se zval pval #> 1: -0.0308 0.0063 -4.9057 <.0001 #> 2: -0.0032 0.0092 -0.3471 0.7285 #> #> Omnibus Test of Hypotheses: #> QM(df = 2) = 33.9192, p-val < .0001 #>### corresponding likelihood ratio test anova(res1, res2)#> #> df AIC BIC AICc logLik LRT pval QE tau^2 #> Full 4 23.2922 25.5520 28.2922 -7.6461 28.3251 0.0269 #> Reduced 2 29.3302 30.4601 30.5302 -12.6651 10.0379 0.0066 152.2330 0.2800 #> R^2 #> Full #> Reduced 90.3948% #>#> #> Hypothesis: #> 1: intrcpt + 35*ablat + 1970*year = 0 #> #> Results: #> estimate se zval pval #> 1: -0.7533 0.0837 -9.0039 <.0001 #> #> Test of Hypothesis: #> QM(df = 1) = 81.0700, p-val < .0001 #>### use predict() to obtain the same linear combination (with its CI) predict(res2, newmods=c(35,1970))#> #> pred se ci.lb ci.ub pi.lb pi.ub #> -0.7533 0.0837 -0.9173 -0.5893 -1.1142 -0.3925 #>### mixed-effects model with three moderators res3 <- rma(yi, vi, mods = ~ ablat + year + alloc, data=dat, method="ML") res3#> #> Mixed-Effects Model (k = 13; tau^2 estimator: ML) #> #> tau^2 (estimated amount of residual heterogeneity): 0.0329 (SE = 0.0272) #> tau (square root of estimated tau^2 value): 0.1814 #> I^2 (residual heterogeneity / unaccounted variability): 33.24% #> H^2 (unaccounted variability / sampling variability): 1.50 #> R^2 (amount of heterogeneity accounted for): 88.24% #> #> Test for Residual Heterogeneity: #> QE(df = 8) = 26.2030, p-val = 0.0010 #> #> Test of Moderators (coefficients 2:5): #> QM(df = 4) = 31.2609, p-val < .0001 #> #> Model Results: #> #> estimate se zval pval ci.lb ci.ub #> intrcpt -10.0824 23.9973 -0.4201 0.6744 -57.1162 36.9514 #> ablat -0.0277 0.0071 -3.9298 <.0001 -0.0416 -0.0139 *** #> year 0.0053 0.0122 0.4371 0.6620 -0.0185 0.0292 #> allocrandom -0.2826 0.2540 -1.1127 0.2658 -0.7804 0.2152 #> allocsystematic -0.1104 0.2551 -0.4330 0.6650 -0.6104 0.3895 #> #> --- #> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 #>### test the 'alloc' factor anova(res3, btt=4:5)#> #> Test of Moderators (coefficients 4:5): #> QM(df = 2) = 1.4674, p-val = 0.4801 #>### instead of specifying the coefficient numbers, grep for "alloc" anova(res3, btt="alloc")#> #> Test of Moderators (coefficients 4:5): #> QM(df = 2) = 1.4674, p-val = 0.4801 #>############################################################################ ### an example of doing LRTs of variance components in more complex models dat <- dat.konstantopoulos2011 res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat) ### test the district-level variance component res0 <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat, sigma2=c(0,NA)) anova(res, res0)#> #> df AIC BIC AICc logLik LRT pval QE tau^2 #> Full 3 21.9174 27.9394 22.3880 -7.9587 578.8640 NA #> Reduced 2 37.6910 41.7057 37.9218 -16.8455 17.7736 <.0001 578.8640 NA #>### test the school-level variance component res0 <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat, sigma2=c(NA,0)) anova(res, res0)#> #> df AIC BIC AICc logLik LRT pval QE tau^2 #> Full 3 21.9174 27.9394 22.3880 -7.9587 578.8640 NA #> Reduced 2 68.4330 72.4476 68.6637 -32.2165 48.5155 <.0001 578.8640 NA #>#> #> df AIC BIC AICc logLik LRT pval QE tau^2 #> Full 3 21.9174 27.9394 22.3880 -7.9587 578.8640 NA #> Reduced 1 439.7236 441.7310 439.7991 -218.8618 421.8062 <.0001 578.8640 NA #>