Yu Junhong

New paper in Imaging Neuroscience by Junhong Yu, Yi-Sheng Wong, and Charly Hugo Alexandre Billaud:

Thought contents during rest account for functional connectivity-behavior associations

doi.org/10.1162/IMAG...

Appreciate the sharing of my lengthy thread. Yes the retraction was not due to transparency, i could have worded that a little better. We've submitted supplementary analyses to show the inclusion (somewhat unorthodox manner) of the previously excluded studies yielded the same significant cluster

We hope that future ALE meta-analyses will be similarly, if not more, transparent about the exclusions, even if this may potentially lead to corrections/retractions. Considering that meta-analyses wield immense influence, these corrections/retractions are necessary for the field to advance.

As you can imagine, this retraction is devastating to us, but we do not regret being transparent about the flaws in this study, even if it has led to the study's retraction.

We do not provide a list of excluded studies, but the excluded studies are cited and mentioned in the results section. Since this meta-analysis was carried out using the ALE GUI, there wasn’t any analysis code we can provide.

Other minor issues: regarding the inclusion of the yoga study, we have explained in great detail within the introduction that yoga incorporates a substantial mindfulness component (e.g., mindful breathing).

This was actually an issue we've pointed out in the retracted study, and also one of rationale for doing an RCT-only meta-analysis

Cyril used a different checklist possibly because their meta-analysis, was not an RCT meta-analysis. They mixed (mostly) cross-sectional comparisons between meditators and non-meditators with a few RCTs. Despite this, they concluded ‘mindfulness meditation practice does induce grey matter changes’

If we had used the checklist included in Cyril et al.’s meta-analysis (another paper linked by Eiko in his PubPeer comment), we would have uncovered some selection and information bias.

We hope that future ALE meta-analyses will be similarly, if not more, transparent about their exclusions, even if this may potentially lead to corrections/retractions. Considering that meta-analyses wield immense influence, these corrections/retractions are necessary for the field to advance.

As you can imagine, this retraction is devastating to us, but we do not regret being transparent about the flaws in this study, even if it has led to the retraction.

We do not provide a list of excluded studies, but the excluded studies are cited and mentioned in the results section. Since this meta-analysis was carried out using the ALE GUI, there wasn’t any analysis code we can provide.

As for the awkward Risk of Bias figure, that is entirely my fault. I came across a meta-analysis in a psychiatry journal and saw that figure, so I thought it would be good to include it in our meta-analysis. It turns out that the criteria in this risk of bias assessment are so easy to pass.

“running ALE analyses on smaller datasets would require special diligence in assessing and reporting the contributions of experiments to individual clusters.”

The total number of studies included in the supplementary analyses is 16, which still falls short of the 23-study cutoff recommended in a paper that Eiko linked. Though the authors acknowledged that this is not a hard cutoff:

That being said, we tried to defend the original conclusions with a supplementary analyses. We created dummy clusters for the 4 excluded studies, placing them far away from the insula region, and added one new study. Even so, the original cluster remained significant.

Second, we were transparent about these exclusions, acknowledging them and discussing their implications. Many ALE papers (www.nature.com/search?q=ALE...) do not even mention the exclusion of studies with null findings. This is, unfortunately, not the norm in the ALE literature.

First, the retraction note made it sound like we intentionally excluded papers with null findings to p-hack our way to obtaining significant results, which was not the case, as acknowledged by Eiko; this was simply a limitation of the ALE method.

Hi I’m the last author of the retracted study; the first author is my PhD student. I don’t disagree with the retraction note. However, it missed out on some important context that I had written in my author’s response, which was not published with the retraction note. Here’s my side of the story