Keywords: meta-analysis, funnel plot, visual inference, publication bias, small study effects
Webpages:
https://CRAN.R-project.org/package=metaviz,
https://metaviz.shinyapps.io/funnelinf_app/ The funnel plot is widely used in meta-analysis to detect small study effects, especially publication bias. However, it has been repeatedly shown that the interpretation of funnel plots is highly subjective and often leads to false conclusions regarding the presence or absence of such small study effects (Terrin, Schmid, and Lau 2005). Visual inference (Buja et al. 2009) is the formal inferential framework to test if graphically displayed data do or do not support a hypothesis. The general idea is that if the data supports an alternative hypothesis, the graphical display showing the real data should be identifiable when simultaneously presented with displays of simulated data under the null hypothesis. When compared to conventional statistical tests, visual inference showed promising results in experiments, for example, for testing linear model coefficients using boxplots and scatterplots (Majumder, Hofmann, and Cook 2013). With the package
nullabor (Wickham, Chowdhury, and Cook 2014) helpful general purpose functions for visual inference are available within
R. Due to the often uncertain or even misleading nature of funnel plot based conclusions, we identified funnel plots as a prime candidate field for the application of visual inference. For this purpose, we developed the function funnelinf which is available within the
R package
metaviz. The function funnelinf is specifically tailored to visual inference of funnel plots, for instance, with options for displaying significance contours, Egger’s regression line, and for using different meta-analytic models for null plot simulation. In addition, the functionalities of funnelinf are made available as a shiny app for the convenient use by meta-analysts not familiar with
R. Visual funnel plot inference and the capabilities of funnelinf are illustrated with real data from a meta-analysis on the mozart effect. Furthermore, results of an empirical experiment evaluating the power of visual funnel plot inference compared to traditional statistical funnel plot based tests are presented. Implications of these results are discussed and specific guidelines for the use of visual funnel plot inference are given.
References Buja, Andreas, Dianne Cook, Heike Hofmann, Michael Lawrence, Eun-Kyung Lee, Deborah F Swayne, and Hadley Wickham. 2009. “Statistical Inference for Exploratory Data Analysis and Model Diagnostics.”
Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 367: 4361–83.
Majumder, Mahbubul, Heike Hofmann, and Dianne Cook. 2013. “Validation of Visual Statistical Inference, Applied to Linear Models.”
Journal of the American Statistical Association 108: 942–56.
Terrin, Norma, Christopher H Schmid, and Joseph Lau. 2005. “In an Empirical Evaluation of the Funnel Plot, Researchers Could Not Visually Identify Publication Bias.”
Journal of Clinical Epidemiology 58: 894–901.
Wickham, Hadley, Niladri Roy Chowdhury, and Di Cook. 2014.
Nullabor: Tools for Graphical Inference.
https://CRAN.R-project.org/package=nullabor.
