Selection models can handle dependent effect sizes in meta-analyses.

When scientists combine results from many studies to find an overall truth, two big problems often get in the way. First, single studies frequently share multiple related results instead of just one, making the data points depend on each other. Second, journals often publish only statistically significant findings, hiding the rest. This creates selective reporting bias. Researchers looked at step-function selection models, which adjust for missing results, but usually miscalculate uncertainty if they ignore connected data. The researchers tested ways to fix this. They used cluster-robust variance estimation and bootstrap resampling, which repeatedly resamples groups of related results to measure uncertainty accurately. They tested these methods on earlier ego depletion data and across computer simulations. They found that these models yield low-bias estimates of average effect sizes. Furthermore, clustered bootstrap confidence intervals give acceptable coverage when a meta-analysis has at least 30 studies.

Why it matters

Social science meta-analyses often combine studies that report multiple outcomes, creating dependent data. This work shows how to correct for selective reporting bias in such cases, giving researchers a robust tool to assess whether published results are trustworthy. Without it, bias corrections could be misleading or too confident.

Questions answered