Research on Research Institute (RoRI)

“Using metrics to assess researchers can be ‘very dodgy terrain,’” says @jameswilsdon.bsky.social.

Great overview of how research assessment is changing worldwide in @nature.com , featuring recent work by RoRI with the Global Research Council: www.nature.com/articles/d41...

At our UCL seminar, ESRC Exec Chair @stianwestlake.bsky.social said UKRI’s analytic capacity is “pretty limited” and “geared towards compliance need.” Better use of data will be key for stronger investment decisions and to act as the UK’s “strategic brain” for R&D: tinyurl.com/5n742x5w

Very much looking forward to this event!

@metaror.bsky.social @asapbio.bsky.social @rorinstitute.bsky.social @aimosinc.bsky.social @cwts.nl

Research assessment: a round-up for early-career researchers Confused by research evaluation and its potential impact on career progression? Here are some pointers as to how things work in different parts of the world.

Research assessment: a round-up

Excellent survey of the assessment landscape by @hollyelse.bsky.social in @nature.com, including our recent work @rorinstitute.bsky.social with the Global Research Council.

www.nature.com/articles/d41...

@alexrushforth.bsky.social @prussianhippie.bsky.social

New in @nature.com: Funding the Frontier, an AI tool to map & predict research impacts.

The piece cites RoRI's @jameswilsdon.bsky.social, who warns such algorithms risk reinforcing past successes over future potential: www.nature.com/articles/d41...

Yesterday, we hosted Stian Westlake (@ukri.org ESRC) for a seminar at UCL on how analytics & experimentation can strengthen research funding and UKRI’s role as a “strategic brain” for UK R&D. With great responses from Molly Morgan Jones (British Academy) & Sarah Chaytor (UCL).

DOIs may seem like a technical detail, but they’re central to how research outputs are recognised.

In a new blog post, RoRI's @ludowaltman.bsky.social & @andre-brasil.bsky.social discuss why we need good practices for DOI registration for publish-review-curate platforms: tinyurl.com/y632m7kz

Innovation in scientific publishing and its implications for Crossref DOI registration practices - Request for input - Crossref Lots of exciting innovations are being made in scientific publishing, often raising fundamental questions about established publishing practices. In this guest post, Ludo Waltman and André Brasil disc...

We need good practices for DOI registration for publish-review-curate platforms like @metaror.bsky.social and others.

@andre-brasil.bsky.social and I wrote a blog post about this: doi.org/10.64000/76j...

Together with @crossref.bsky.social we hope to receive input from the community on this issue!

🧠 Does STI policy need a strategic brain?

Join us next Tuesday at UCL for a talk by @stianwestlake.bsky.social (Executive Chair, @ukri.org ESRC) on the future of UK R&D, with responses from @britishacademy.bsky.social's Molly Morgan Jones & UCL’s Sarah Chaytor: tinyurl.com/vekzka23

Distributed Peer Review – How the wisdom of the crowd can allocate grant funding - Impact of Social Sciences New research shows engaging the applicants in funding review processes can resolve and reframe many issues facing peer review.

🐝 A move from the wisdom of the gatekeeper to the wisdom of the (expert) crowd

RoRI researchers wrote for @lseimpactblog.bsky.social about our study with @volkswagenstiftung.de, testing distributed peer review as an alternative to traditional funding panels:

Distributed Peer Review – How the wisdom of the crowd can allocate grant funding - Impact of Social Sciences New research shows engaging the applicants in funding review processes can resolve and reframe many issues facing peer review.

💥New: Distributed Peer Review – How the wisdom of the crowd can allocate grant funding

✍️ Melanie Benson Marshall, Anna Butters, @stephenpinfield.bsky.social @tomstafford.mastodon.online.ap.brid.gy

#PeerReview #PRW25 #ResearchPolicy

Does STI policy need a strategic brain? A seminar on how metascience might sharpen investment decisions across the UK’s research system

Does STI policy need a strategic brain?

Join us at 5pm on 30 Sept for a talk by @stianwestlake.bsky.social, Executive Chair, ESRC @ukri

Hosted by STEaPP-UCL & RoRI, w/ responses by @britishacademy.bsky.social’s Molly Morgan-Jones & UCL’s Sarah Chaytor

www.eventbrite.co.uk/e/does-sti-p...

🧠 Does STI policy need a strategic brain?

Join us at UCL on 30 Sep for a discussion with @stianwestlake.bsky.social, Executive Chair of @ukri.org's ESRC, with responses from Molly Morgan Jones (@britishacademy.bsky.social) and Sarah Chaytor (@uclpublicpolicy.bsky.social): tinyurl.com/vekzka23

Join RoRI's Peter Kolarz tomorrow for a discussion on how to build impact into grant applications ⬇️

RoRI's @tomstafford.mastodon.online.ap.brid.gy has shared reflections on his Substack about our recent study with @volkswagenstiftung.de on distributed peer review.

If you’ve read the paper, this is your chance to see the “director’s commentary” - and if not, it’s a great place to start ⬇️

“The discipline of Sociology has taught me that systems usually don’t want to change, and will often resist change even if their longevity depends on it. For the grant-funding system, change is here."

RoRI's Peter Kolarz on the future of research funding: www.scientifyresearch.org/blog/grant-f...

📢 New RoRI paper out in Science & Technology Studies

The study explores how 8 research funders navigate the ambiguous role of “excellence” - not simply as a matter of fact, but increasingly as a matter of concern requiring reconfiguration: sciencetechnologystudies.journal.fi/article/view...

🔓 It's great to see authors sharing their experiences with publishing on MetaROR (MetaResearch Open Review) — our open review platform for metascience using the publish–review–curate model: www.openscience.nl/en/cases/the...

"Much of what shapes a scientific paper never sees the light of day."

RoRI's André Brasil writes about MetaROR (MetaResearch Open Review) and opening the black box of peer review in the inaugural issue of REACH by @sci-integrity.com: www.sci-integrity.com/reach-inaugu...

🇦🇹 How are researchers in Austria doing?

RoRI partner @fwf-at.bsky.social has just published Austria’s largest survey of the research community.

Over 3,300 researchers shared their views on careers, working conditions, and funding: www.fwf.ac.at/en/news/deta...

AI in science: is it useful? Artificial intelligence might be useful to scientists in some circumstances, but it’s unlikely to revolutionise the job, according to presentations at a conference held last month that discussed the t...

#AI is reshaping #science, but not without risks. At Metascience 2025, researchers praised AlphaFold yet warned of spam, fake papers and hype drowning real discovery.
🔎https://sciencebusiness.net/news/ai/ai-science-it-useful

Distributed peer review cuts funding decision time, our new paper shows; but the approach also raises important questions about fairness, bias, and gaming the system.

Great to see our study with @volkswagenstiftung.de featured in @sciencebusiness.net (£): sciencebusiness.net/news/r-d-fun...

Distributed Peer Review: Studie zeigt Potential für schnellere und demokratischere Begutachtung | VolkswagenStiftung Eine neue RoRI-Studie zum Einsatz von Distributed Peer Review (DPR) in der VolkswagenStiftung zeigt, wie das Begutachtungsverfahren Förderentscheidungen beschleunigen und sie demokratischer machen kan...

Eine neue Studie von @rorinstitute.bsky.social zum Einsatz von Distributed #PeerReview (DPR) in der VolkswagenStiftung zeigt, wie das Begutachtungsverfahren Förderentscheidungen beschleunigen und sie demokratischer machen kann. 👇

#Begutachtung #ResearchFunding

Radical method cuts time on peer review by 60%, trial finds - Research Professional News Grant proposals also get more attention under distributed peer review, experiment shows

Great to see our new study on distributed peer review featured in ResearchProfessional (£).

Together with @volkswagenstiftung.de, we tested if asking applicants themselves to review each other’s grant proposals could speed up the funding process: www.researchprofessionalnews.com/rr-news-euro...

Democratising funding evaluation can slash the time it takes to deliver a final decision by over 50% - Research on Research A new RoRI study with the Volkswagen Foundation shows how distributed peer review can make funding decisions more democratic

🚀 New study out!

Together with @volkswagenstiftung.de, we’ve tested distributed peer review - a model where applicants also act as reviewers - to see how it compares with conventional panel review: researchonresearch.org/democratisin...

Over time I expect research communities will develop a better understanding of the pros and cons of different models, like the MetaROR model, the PCI model and others.

We need experimentation to find out what does and doesn't work!

@rorinstitute.bsky.social @aimosinc.bsky.social

PLOS Biology announces agreement to become a MetaROR partner journal - The Official PLOS Blog Note: PLOS issued the following press release on Thursday, August 14th. SAN FRANCISCO, CA — Today, PLOS Biology announced a new agreement…

We are proud to announce a new agreement with @rorinstitute.bsky.social and @aimosinc.bsky.social to become a partner journal with MetaResearch Open Review (MetaRoR) @metaror.bsky.social

#metaresearch #openscience 🧪

Learn more: plos.io/45wPZB9

From its historical roots to today’s diverse, decentralised movement, we explore five key vectors shaping the future of metascience - and what they mean for research systems worldwide: transforming-evidence.org/blog/what-is...

The plenary sessions from the Metascience 2025 Conference are now available to watch online!

Three days, dozens of speakers, and conversations that will shape the future of research on research. Check out the recordings on Cassyni here: cassyni.com/s/metascienc...

The effect of APC discounts on Ukraine's participation in gold open access journals

🔒Our latest study regards "the effect of APC waivers on the participation of Ukrainian researchers in fully Gold OA journals".

The reviewers call it "well-developed", "timely" & "important" while questioning the focus on only 5 publishers.

👇Read the assessment, reviews & author response on MetaROR