Marta Topor

Does anyone have experience with CADIMA for screening articles? www.cadima.info I’d be interested in hearing thoughts, especially if you can compare it to rayyan

#evidencesynthesis #systematicreview

*hundreds of participants 😅
Was a tad over excited there

The key was a very well set up Github repo with all data, metadata, R project with all scripts and a description of results in Rmd files. I put a lot of effort into it mainly because I was learning a lot of new stuff and wanted to make sure it's as clear as possible.
I am so grateful to myself!

I abandoned a paper 1.5 yrs ago.😵‍💫
I picked it up today and reran the analyses, probably my most complicated to date - 3 different Bayesian regression models in Stan, thousands of participants, multiple R scripts.
Everything reproduced perfectly 🥲 What a moment

What's next?
Let's try to unify the field a bit more.

We're already taking a step into the future with the CO-IND (Cognition and Intellectual Disability) project. 37 groups in 23 countries using the same basic research protocol. Lots of opportunities ahead.

coind-project.github.io

❗❗❗ 10 years ago, it was still uncommon to seek an ethics review board approval for this type of studies

Single-case designs are the most common in the field. Both single-case and group design studies tend to use unstandardised tests, which makes it difficult to compare outcomes across studies. Or to replicate the studies if insufficient information about the measures is shared in the report.

In 940 reports, we were able to detect only a handful of theories mentioned in the background.
Most interventions were based on previous interventions, and no, these were not replications.
Studies tend to present new interventions/ideas on a small scale, making it hard to scale up and generalise.

@metahag.bsky.social had a paper published and I can't help myself from shouting about it! 📣

A scoping review, 952❗educational interventions, pupils with intellectual disability.

Found: lacking theoretical background and ethics board approvals, use of unstandardised tests
doi.org/10.1016/j.ij...

Our lab has the capacity to test ~500 uni students each semester
If you’re a researcher in cognitive psychology or metascience and need data collection support, we’d love to collaborate. We can help collect high-quality data from a large student sample.
Get in touch to discuss potential projects!

Very relevant following the keynote by Leif Nelson! #PSE8

🫶 Thank you, Jade, it's so great to have you as one of my closest collaborators, seeing the journey from pretty much the very beginning!

@lakens.bsky.social any chance you might have recommendations for a nice shortish running route? :)

Fab!

My intial idea is something like 5k, easy pace of ~6m/km :) But I haven't mapped it, so not sure where exactly

Going to the 8th Perspectives on Scientific Error Workshop. #PSE8
Taking my running shoes and planning to go for a sightseeing run. Anyone else keen to explore the city like this? 👟🏃‍♀️🏙️

As of today, on the @pci-regreports.bsky.social home page, you will find my name among amazing researchers who have been a real inspiration at different stages of my researcher journey.

I'm so grateful for this opportunity 🙏
I really enjoy the recommender role & I'm ready to take things up a notch

RegCheck RegCheck is an AI tool to compare preregistrations with papers instantly.

Comparing registrations to published papers is essential to research integrity - and almost no one does it routinely because it's slow, messy, and time-demanding.

RegCheck was built to help make this process easier.

Today, we launch RegCheck V2.

🧵

regcheck.app

If you have any tips to efficiently manage communication and problem-solving in #BTS projects, do share!
In our project, most groups are now preparing for data collection, and this means lots of uncertainties and urgent questions.

Tip 3: I have told all collaborators to account for potential delays due to my not being able to answer all emails and questions right away. I know that this can get frustrating sometimes, but it's best to manage expectations early on.
This solution has helped to take some pressure off of me.

If no one comes during the on-call hours, I tackle the emails in the project-specific mailbox.

Tip 2: I scheduled regular on-call hours for collaborators. Once a week, I'm on Zoom for 2 hours. Instead of sending an email, colleagues can join the call and ask questions. The first one was yesterday, and indeed, it did save me answering an email and issues were fixed right away.

Tip 1: I have told all collaborators to send emails to a separate mailbox. This way, it will be much easier for me to sort through project-related matters, and it will be less likely that these emails get lost in my main mailbox.

For context: We are 37 research groups in 23 countries. Our participants are children and young people with intellectual disability.
We have a complex protocol adapted to different cultural contexts, 23 languages and 2 modes of data collection (in-person and remote).
coind-project.github.io

Today is the first day I managed to reach <20 unread emails in my mailbox. In ~3 months.
This is big.
I'm managing a Big Team Science project #BTS and
normally sit with 50-70 unread emails at a time.
Here are some solutions I am trying for better communication and mailbox hygiene in 2026.

⬇️A metascience collab developing an integrity screening tool. Tool testers needed!
You're a researcher bothered by integrity issues? Join us! The team: @metahag.bsky.social @steamtraen.eu @jamiecummins.bsky.social @adrienfillon.bsky.social @jamesheathers.bsky.social @samwestwood.bsky.social & others

Looking for user panelists to contribute to the Stage 2 of our registered report: Developing and Piloting a QUick Evaluation of Research Integrity for Evidence Synthesis in Educational Research (QUERIES-Edu). osf.io/tc5n3

If you’re interested, sign up here: forms.gle/SSmPmaCrdGvnHPdx9

GRIM-U: A GRIM-like observation to establish impossible p values from ranked tests - Medical Evidence Project The Medical Evidence Project uses forensic metascientific methods to examine medical research publications that have a disproportionately high impact on human health. Our aim is to determine where pro...

First MEP report.

Medical research often uses non-parametric tests. We looked, and noticed that small-sample rank-based tests (i.e. M-W, Wilcoxon) have significant granularity... like p-values!

So, here's GRIM for U values -- GRIM-U.

Enjoy.

medicalevidenceproject.org/grim-u-obser...

🏋️‍♀️ New challenges:

1) Promoting a wider adoption of open and reproducible research practices at the institutional level, in a new role as the Open Science Coach at Linköping University.

2) I'm joining the PCI RR Managing Board, and really looking forward to it! @pci-regreports.bsky.social

3/3

🤝 As one of the co-founders, I' proud of all the progress we have made! I'm both sad about leaving but also excited about what comes next. I'll stay in the loop and collaborate with the DKRN as a representative of the Swedish community of researchers focusing on open and reproducible research.

2/3