Testing for Questionable Research Practices in a Meta-Analysis: An Example from Experimental Parapsychology

The question behind the meta-analysis

A Ganzfeld experiment asks a participant to identify a randomly chosen target from four possibilities. Chance performance is therefore 25%. The paper examines whether the higher hit rates in a published database could arise from questionable research practices: choices in collecting, analysing or publishing data that can make an effect look stronger.

Simulating the research process

The simulations reproduce the numbers of studies and trials in the selected database. They model practices including publication bias, optional stopping and biased exclusion of participants. A genetic algorithm searches for combinations that match several features of the evidence, including hit rates, the relationship between sample size and hit rate, and the distribution of p-values.

The authors compare unrestricted models with models whose assumptions are constrained by surveys of research practices in psychology. This makes the assumptions themselves part of the analysis, rather than treating a single correction for publication bias as sufficient.

What the findings support

The modeled practices explain part of the reported effect, but do not fully reproduce the database within the chosen constraints. That remainder could reflect an anomalous effect or research practices the model does not include. The paper also identifies an untested assumption: that the prevalence of these practices is comparable in psychology and parapsychology. Its broader contribution is a way to examine how methodological choices influence a meta-analysis; it does not establish the cause of the remaining effect.