AI tool detects LLM-generated text in research papers and peer reviews
Authors and peer reviewers are failing to disclose the use of LLMs despite journal policies limiting their use.
@theaacr.bsky.social found that 23% of abstracts in manuscripts and 5% of peer-review reports submitted to its journals in 2024 contained text that was probably generated by large language models (#LLMs)
www.nature.com/articles/d41...
#AI #metaresearch
12.09.2025 13:44 — 👍 4 🔁 4 💬 0 📌 0
Automated verification of citations could help journals and peer reviewers identify questionable citation practices and reduce propagation of misinformation, improving the trustworthiness of scientific evidence.
In the #PubMedCentral-OA dataset, 34% (981 of 2898) of citation quotations were erroneous. #LLMs (@GeminiApp @GPT4oGPT) can be trained to detect citation errors with moderate accuracy, according to @uofilsystem.bsky.social researchers
#AI
05.09.2025 20:57 — 👍 3 🔁 1 💬 0 📌 0
The annotation process underscored the subjective nature of assessing promotional language, particularly for context-dependent adjectives like emerging and latest, and the need for refined constructs to capture gradations of promotional language. Despite these limitations, the pilot study indicates the potential for machine learning models trained on well-annotated datasets to contribute to the automated detection of hype. Future steps include modifying the annotation systems and experimenting with large language models under zero/few-shot regimes.
A pilot study testing the ability of #MachineLearning models trained on well-annotated datasets to automate detection of hype in biomedical research writing (eg ‘groundbreaking', ‘revolutionary’) showed 70-80% accuracy vs human detection
peerreviewcongress.o...
05.09.2025 21:14 — 👍 5 🔁 1 💬 0 📌 0
Detecting LLM-Generated Peer Reviews
The integrity of peer review is fundamental to scientific progress, but the rise of large language models (LLMs) has introduced concerns that some reviewers may rely on these tools to generate...
Methods to detect #LLM-generated #PeerReview: in ms, add command to LLM to:
1. Watermark (insert a random start string, fake citation, or technical term)
2. Hidden prompt (insert white-colored text, different language, font manipulation)
@cmu.edu
05.09.2025 21:44 — 👍 4 🔁 2 💬 0 📌 0
LLM-generated reviews matched or exceeded human reviewers on a few key dimensions of review quality. A fuller analysis will shed more light on the potential value of LLM peer reviews and how they could complement human peer reviewers’ work.
Comparing #LLM vs human #PeerReview of 5 @bmj.com mss, LLM reviews matched or exceeded humans on key elements of identifying strengths/weaknesses, commenting on writing, organization, and Tables and Figures, and constructiveness
peerreviewcongress.o...
05.09.2025 21:45 — 👍 5 🔁 3 💬 0 📌 0
#ZakKohane on #AI in #PeerReview: the surge in scientific publication is straining the conventional review system beyond capacity. @ai.nejm.org piloting AI-augmented fast track review: manuscripts w high prior of being good gets assigned to #GPT5 + a human, disposition reviewed by editorial team!
05.09.2025 13:27 — 👍 8 🔁 4 💬 0 📌 1
Detection of #AI-generated text at 10 @theaacr.bsky.social journals increased Q1 '23 after public release of ChatGPT; a policy prohibiting its use slowed detections in reviewer comments. Presence in mss was assoc'd w a higher rejection rate before peer review @evanko.bsky.social #cancer
03.09.2025 14:35 — 👍 6 🔁 2 💬 1 📌 1
The effect of our intervention was not statistically significant (mean citation rate, −0.007; 95% CI, −0.055 to 0.041) and held for all prespecified sensitivity analyses.
#RetractoBot auto-emails authors when papers they've cited are retracted. An RCT comparing it vs no notice showed no diffs in 1y citations
@bengoldacre.bsky.social @ndevito1.bsky.social @bennettoxford.bsky.social
@ox.ac.uk @oxprimarycare.bsky.social
www.retracted.net/
cc: @retractionwatch.com
03.09.2025 16:25 — 👍 11 🔁 4 💬 1 📌 0
The PNN involved 56 journals across 12 publishers, with Springer Nature (n = 29), Elsevier (n = 26), and Bentham Science Publishers (n = 25) having the most articles involved out of the total (n = 123). The involved
authors were affiliated with 40 countries and 232 organizations.
The #Pharmakon Neuroscience Network, involving 361 authors from 40 countries in 123 articles/56 journals from 12 publishers 2019-2025, appears to have grown too rapidly to have developed naturally
@fauxsci.bsky.social @helenedraux.bsky.social @digital-science.com
(see also: doi.org/10.48550/arX...)
03.09.2025 16:45 — 👍 4 🔁 2 💬 1 📌 1
Unexplained textual similarities (eg, the same unlikely typo appearing in dozens of articles) common to many of these articles suggest the involvement of paper mills, organizations that mass produce, sell authorship on, and publish fraudulent scientific manuscripts at scale.
Scanning electron microscope image (SEM) metadata did not match the SEM manufacturer or model listed in the text in 21.2% of 11.3K materials science and engineering articles, a potential indicator of irreproducibility.
@reeserichardson.bsky.social
Read the full report at journals.plos.org/pl...
03.09.2025 20:50 — 👍 3 🔁 2 💬 1 📌 0
Gel blots (eg, Western blots) were the most frequently cited image type in problematic retractions, appearing in 1074 articles (51.68%). Between-article image reuse, where an image and its associated data are duplicated across different publications, was the most common context of misuse, identified in 1241 cases (59.72%). Notably, 982 retractions (47.28%) were attributed to paper mills. Image duplication was the predominant cause of retraction, accounting for 1827 cases (87.92%). Only 1 retraction was attributed to computer-or artificial intelligence–generated manipulation.
Among 2078 retracted articles flagged for problematic image manipulation by @retractionwatch.com, gel blots were the most frequently cited image type (52%); image duplication was most common retraction cause (88%)
@phillcardenuto.bsky.social @loyolachicago.bsky.social @unicampoficial.bsky.social
03.09.2025 20:30 — 👍 6 🔁 4 💬 1 📌 0
International Congress on Peer Review and Scientific Publication
Enhancing the quality and credibility of science.
Unraveling Spin & Selective Reporting in Medical #AI Research: A Cross-Sectional #MetaResearch Study
from #UNSWSydney researchers
is a poster presentation all 3 PRC days
Register to stop by, ask questions, say hello!
Learn more and register at peerreviewcongress.org
@jama.com @bmj.com
06.08.2025 20:25 — 👍 3 🔁 2 💬 0 📌 0
International Congress on Peer Review and Scientific Publication
Enhancing the quality and credibility of science.
Authorship and Contributorship Criteria and Practices at Annals of African Surgery
from EIC James Kigera and AAS colleagues
is a poster presentation all 3 days of the PRC
Register to stop by, ask questions, say hello!
Learn more at peerreviewcongress.org
#AfricanScience
@jama.com @bmj.com
07.08.2025 00:47 — 👍 2 🔁 1 💬 0 📌 0
International Congress on Peer Review and Scientific Publication
Enhancing the quality and credibility of science.
Development of a Tool for Addressing #COIs in Trials (TACIT) for Use in Systematic Reviews
from @alundh.bsky.social @isabelleboutron.bsky.social
@cochranedkcebmo.bsky.social @upcite.bsky.social
is a PRC poster presentation
Register to stop by, ask the authors questions!
peerreviewcongress.org
09.08.2025 13:55 — 👍 5 🔁 4 💬 0 📌 0
Newsletters | Cochrane Bias
The latest Bias Methods Group newsletter is live! Dive into fresh insights and essential updates on tackling bias in research, as well as a call for abstracts for the 2025 annual meeting! Read it on the BMG website: methods.cochrane.org/bias/about-u...
19.06.2025 09:01 — 👍 4 🔁 4 💬 0 📌 0
2025 annual meeting(s) and call for abstracts | Cochrane Bias
Are you interested in presenting your research at the annual Bias Methods Group meeting? If so, please do not forget to submit your abstract to cochranemethodsbias@gmail.com by August 11. For more information, please click here: methods.cochrane.org/bias/news/20...
19.06.2025 09:04 — 👍 2 🔁 2 💬 0 📌 0
Two modules of the International Summer School in Public Health came to their conclusion yesterday. @cressumr1153.bsky.social and Institut Paris Public Health. After two days of intensive exchanges, workshops and teaching, these modules came to a close in a constructive and collaborative spirit.
12.06.2025 08:45 — 👍 1 🔁 1 💬 0 📌 0
A special shout out to:
@rohitarorayyc.bsky.social
@paullusla.bsky.social
@jzsang.bsky.social
for working tirelessly to make this dream happen <3
And our advisors:
@isabelleboutron.bsky.social
David Moher
@geochurch.bsky.social
@nikbobrovitz.bsky.social
12.06.2025 17:04 — 👍 3 🔁 1 💬 1 📌 0
Excellent opportunity to learn innovative design, to interact with outstanding researchers in the field of epidemiology and statistics!!!
The international summer school of the @cressumr1153.bsky.social in the centre of Paris
04.06.2025 17:42 — 👍 3 🔁 0 💬 0 📌 0
Rethinking the role of suffering in healthcare: Reflections on the interdisciplinary study day of March 20, 2025 - CRESS - Centre de Recherche en Epidémiologie et Statistiques
On March 20, Astrid Chevance of the METHODS team at @cressumr1153.bsky.social and Luz Ascarate (philosopher) organized an interdisciplinary study day entitled “Suffering and pain”.
cress-umr1153.fr/en/rethinkin...
@inserm.fr
@upcite.bsky.social
@univ-spn.bsky.social
@inrae-france.bsky.social
19.05.2025 07:35 — 👍 2 🔁 1 💬 0 📌 0
French trial sponsors are urged to bolster clinical trial transparency
A French government working group recommended a sweeping plan to publicize study data and issue guidelines for reporting results on a timely basis.
"French trial sponsors are urged to bolster clinical trial transparency": STAT News covers the recent French report urging that study results be published in a registry. I was very happy to be interviewed and provide some context.
15.05.2025 14:20 — 👍 2 🔁 1 💬 1 📌 1
#SemainedelaRechercheClinique 🔬 | @isabelleboutron.bsky.social mène des travaux visant à améliorer la conception, la transparence et l'interprétation des essais cliniques. Découvrez sa vision⬇️
@ap-hp.bsky.social @upcite.bsky.social @cochranefrance.bsky.social @cressumr1153.bsky.social @inserm.fr
13.05.2025 12:55 — 👍 7 🔁 2 💬 0 📌 0
YouTube video by CRESS UMR1153
Portrait du CRESS - Pierre-Yves Ancel
Pierre-Yves Ancel, co-founding member of @cressumr1153.bsky.social, looks back at the history of the center created in 2014. An ambitious project to foster interdisciplinarity in response to public health challenges.
youtu.be/a4tv4VY5uE8
👉 Subscribe for more videos!
@upcite.bsky.social
12.05.2025 09:42 — 👍 1 🔁 1 💬 0 📌 0
New post on ResToRes's website "Massive collaboration sharpens tools for identifying problematic trials". It is about the recent publication of INSPECT-SR Stage 2 findings in The Journal of Clinical Epidemiology.
restores.univ-rennes.fr/highlights/m...
12.05.2025 12:39 — 👍 3 🔁 1 💬 0 📌 0
International Public Health Summer School: From June 2 to June 13, 2025 - CRESS - Centre de Recherche en Epidémiologie et Statistiques
'Early bird' rates extended!
To allow as many people as possible to experience the unique and enriching International Summer School in Public Health we've decided to extend our early bird rates until May 19, 2025, offering a learning opportunity open to everyone!
cress-umr1153.fr/en/summer-sc...
22.04.2025 08:42 — 👍 1 🔁 1 💬 1 📌 0
Cochrane Bias Methods Group: raising awareness of methods to detect biases in the design, conduct and reporting of health research studies.
Behavioural Scientist
illusions of truth | behaviour change | meta-science | Registered Reports lover | transparency & openness in research | responsible researcher assessment | 1st gen | she/her 🌱😻
Leading advocate for preprints, integrity, community & improved research culture | Host Preprints in Motion podcast | Immunologist | https://linktr.ee/jacoates
Bsky archive of Official Horizon EU profile for the EU's Research & Innovation Programme.
europa.eu/!Xf47NRP
check out also @creativeeurope.bsky.social and @euerasmusplus.bsky.social
atm powered by @gogash.bsky.social
EU regulatory authority working for public and animal health.
We ensure that all medicines available on the EU market are high quality, safe and effective. Based in Amsterdam.
https://www.ema.europa.eu/
Health Economist | Evidence Synthesis & Metanalysis | Clinical Guideline Methodologist
📍Cadre de santé et masseur kinésithérapeute
🧑🎓📈📊 Actuellement PhD en D2 à Lille. Thème : Isocinétisme, genou et rachis.
💼DU éthique en santé, licence ès sciences de l'éducation, M2 recherche en rééducation
🎶🎸🎤Musicien
Le tout, c'est en partie moi-même 😉
Pharm.D, étudiant en médecine lVice-président @upcite.bsky.social | #ESR #VieÉtudiante @circleu.bsky.social 🇫🇷🇪🇺
Des données probantes, des décisions éclairées, une meilleure santé.
france.cochrane.org
📍 Postdoc @ CRESS, Université Paris Cité
Clinical Epidemiology | Youth Mental Health
Person-Centred Care | Measurement Science
population health scientist, gp, bass, and bicycles
Associate Professor in Biostatistics, Oslo Center for Biostatistics and Epidemiology @ocbe.bsky.social, University of Oslo | https://www.cer-methods.com/
Dr of Public Health & Information Scientist @ German university clinic and German research institute - Co-Lead #PlanetaryHealth Group @ global charity - Editorial Board @ journal - Activist @ Earth.
Bienvenue sur le compte officiel de l'Université Sorbonne Paris Nord !
www.univ-paris13.fr
Bienvenue sur le compte d'INRAE
Institut national de recherche pour l’agriculture, l’alimentation et l’environnement
Compte officiel. Université Paris Cité, Illuminons le monde de demain. #EnlightenTheFuture #IlluminonsLeMondeDeDemain ✨
Encouraging research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.
peerreviewcongress.org.
Associate professor in digital rehabilitation at CRESS, University of Paris Cité, Physiotherapist.
Father, editor [@elife.bsky.social], rower. Views my own