Olmo van den Akker

Introducing the Simulation Studies Preregistration Template: Q&A with Björn S. Siepe, František Bartoš, and Samuel Pawel Initially submitted through COS’s open call for community-designed preregistration templates, the Simulation Studies Template is now part of the expanding collection of preregistration resources available on the OSF.

Simulation studies are widely used, but have historically lacked clear guidance around preregistration.

In response to this gap, a team of researchers have developed a new template for preregistering simulation studies, now available on OSF.

🚀 Read our Q&A: www.cos.io/blog/intr...

Easily download files from the Open Science Framework with Papercheck Researchers increasingly use the Open Science Framework (OSF) to share files, such as data and code underlying scientific publications, or ...

New blog post: Easily download files from the Open Science Framework with Papercheck daniellakens.blogspot.com/2025/07/easi...

Downloading all files in an OSF project can be a hassle. Not anymore, with Papercheck! Just run:

osf_file_download("6nt4v")

Peer Review at the Crossroads

When MetaROR launched at the end of last year, we published four articles. One of these has now returned to our platform for another round of peer review.

👇 Read the editorial assessment, reviews & full manuscript text at MetaROR (incl. links to the first version and first round of reviews)

[1/3]

figure showing nine data points (each for a distinct publisher) in a graph where Y-axis is our open science index and X-axis is academia friendliness (=proportion of published journals co-owned by an academic entity); strong positive relationship

Non-profit publishers (PLoS, Oxford, Cambridge) and publishers partnering a lot with academia (Wiley) are much more committed to open science than the others - MDPI being the worst by large. (4/5)

Registration now open for this big meta-science unconference in Berlin, hosted by the QUEST Center for Responsible Research!

Take your pick from 10 projects to collaboratively work on a tangible output that will make science better: rr-in-action2025.org/programme/

#metascience2025

NB: The paper is not about whales

Transparency in the secondary use of health data: assessing the status quo of guidance and best practices | Royal Society Open Science We evaluated what guidance exists in the literature to improve the transparency of studies that make secondary use of health data. To find peer-reviewed papers, we searched PubMed and Google Scholar. To find institutional documents, we used our personal ...

Nice! We recently published a review on the guidance that is available for researchers to make their secondary analyses more transparent, might be of interest :-)

royalsocietypublishing.org/doi/full/10....

Responsible Research | In Action - An Unconference exploring the world of responsible research with Responsible Research in Action. Learn about ethical practices, principles, and innovations in impactful science

Shape research culture + practices! Join the Responsible Research in Action Unconference 2025, Sept 22–24, Berlin, to collaborate on innovative solutions – from reproducibility checks to researcher-driven funding models.
rr-in-action2025.org
@dfg.de @ucoimbra.bsky.social @berlinualliance.bsky.social

Be part of the future and submit your metascience work to MetaROR!

#metascience2025

We had a great time at our session at #metascience2025 on Monday! We got a lot of new members and connections that joined our community over the last days.

Missed our talk, and want to join PYMS or learn more? Find one of our board members at the last day of the conference!

@metaror.bsky.social is a scholar-controlled peer review service for metascience research. Check us out! Metaror.org
#metascience2025 (I am an associate editor at MetaRoR) @ludowaltman.bsky.social @aidybarnett.bsky.social

Yes, true.

Limitation: It was a simulated review environment and participants were academics interested in the startup running the study. Unclear how this generalizes to actual peer review at scientific journals.

Taking that into account, I do think the study shows that there is potential for human-AI reviews

Conclusion 2: humans found it very difficult to distinguish AI reviews from human reviews, although graduate students were much better at it than other participant groups

Hot topic at the moment: AI in peer review.

Ashia Livaudais from startup SymbyAI discussed this during a session on peer review at #metascience2025

Conclusion 1: the reviews of human-AI combinations were rated higher in quality than reviews of humans or AI by themselves

Would love to hear more about this! Where can I find them?

This is only focused on meta-research relevant to biology, right? Or are you also open to wider applications?

Yes! Curious to hear how RegCheck is developing.

PYMS has a new board and it is booming! Don't miss this session in London if you are an early career meta-scientist (or are interested in meta-science careers)!

Screenshot of the content of the Snakes and Ladders Metascience panel

We are looking forward to Metascience 2025 in London next week!

Especially to the panel where we will discuss PYMS! Come discuss career paths with us Monday 14:00 - 15:30:

Session 2, Panel T2.3; Snakes and ladders: mapping research careers

Multi-Stage Registration: A Better* Way to Pre-Register — Meta-Research Center This blogpost was written by Cas Goos. Cas his PhD project focuses on studying and enhancing various interventions for improving scientific robustness at journal level, supervised by Michèle Nuijten a...

What if you want to deviate from you preregistration but keep the label "confirmatory"? Cas Goos introduces multi-stage registration as a solution.

Read all about it in the semi-new (I somehow missed a month of blogging?) blog post from the Meta-Research Center:

metaresearch.nl/blog/2025/5/...

📣New Preprint Alert!

Esther Maassen meticulously simulated the effect of p-hacking and publication bias on effect size & heterogeneity estimates.

💡bad: selective outcome reporting & optional dropping
💡bad: publication bias
💡not so bad: optional stopping/outlier removal

osf.io/preprints/ps...

I guess they are becoming aware of their own irrelevance and want to nip initiatives like @peercommunityin.bsky.social and @metaror.bsky.social in the bud.

How about solving one? 😀

The Future of Scientific Publishing - survey

We are reviewing the current state of scientific publishing, and we want to hear from researchers on their perceptions of publishing in the current context and how it might develop in future. Complete our short survey now:

The program for the PYMS Summer Symposium. Text-format program including abstracts can be found via this link: https://tinyurl.com/muaa5dbd

Program finalized!

We're very excited to announce our program for the PYMS Summer Symposium 2025.

Check out the abstracts here: tinyurl.com/muaa5dbd

Don't want to miss out? You can still sign up until the end of this week (23rd of May): forms.gle/yvgVBf8VKCFu...

Sage Journals: Discover world-class research Subscription and open access journals from Sage, the world's leading independent academic publisher.

Out now: our paper on how to deidentify data sets in the social sciences while navigating privacy concerns. The paper includes a step-by-step de-identification guide and two worked examples, with tons of accompanying material on OSF. journals.sagepub.com/doi/10.1177/...

With all due respect to Cathleen's excellent reporting (must read imo), I'd say the thing being most exploited here is our failure to insist on open, expert peer-review, particularly from journals that claim to be peer-reviewed.

The PCI Psychology website is live! psych.peercommunityin.org We're working hard to be ready for launch (planned May 2025) but there's plenty you can do in the meantime to help it along. Go check it out! #psychscisky #scipub