Remember me

Register  |   Lost password?


arXiv logo for blog page

Economically rational sample-size choice and irreproducibility. (arXiv:1908.08702v1 [econ.GN])

Sun, 25 Aug 2019 23:01:59 GMT

Several systematic studies have suggested that a large fraction of published
research is not reproducible. One probable reason for low reproducibility is
insufficient sample size, resulting in low power and low positive predictive
value. It has been suggested that insufficient sample-size choice is driven by
a combination of scientific competition and 'positive publication bias'. Here
we formalize this intuition in a simple model, in which scientists choose
economically rational sample sizes, balancing the cost of experimentation with
income from publication. Specifically, assuming that a scientist's income
derives only from 'positive' findings (positive publication bias) and that
individual samples cost a fixed amount, allows to leverage basic statistical
formulas into an economic optimality prediction. We find that if effects have
i) low base probability, ii) small effect size or iii) low grant income per
publication, then the rational (economically optimal) sample size is small.
Furthermore, for plausible distributions of these parameters we find a robust
emergence of a bimodal distribution of obtained statistical power and low
overall reproducibility rates, matching empirical findings. Overall, the model
describes a simple mechanism explaining both the prevalence and the persistence
of small sample sizes. It suggests economic rationality, or economic pressures,
as a principal driver of irreproducibility.