Andrey Lovakov

I scraped unpublished manuscripts from bioarxiv and injected half of them with a hidden prompt.

I then asked three LLMs to review the paper - in half cases I told the LLM to check for prompt injections.

📈 Prompt injections lead to more recommendations
📉 Prompt checks eliminated this effect

Generalization bias in large language model summarization of scientific research | Royal Society Open Science Artificial intelligence chatbots driven by large language models (LLMs) have the potential to increase public science literacy and support scientific research, as they can quickly summarize complex sc...

Generative AI is nearly 5 times less accurate than humans.

"In a direct comparison of LLM-generated and human-authored science summaries, LLM summaries were nearly five times more likely to contain broad generalizations (OR = 4.85, 95% CI [3.06, 7.70])."
royalsocietypublishing.org/doi/10.1098/...

The brand new `legendry` package by @teunbrand.bsky.social takes ggplots2 guides to the next level 🤩

teunbrand.github.io/teunbrand_bl...

Next week we will have another seminar discussing academic geopolitics - this time from Kazahstanian perspective!

15th of May, 16:00 CET - see you on Zoom!

Register: us06web.zoom.us/meeting/regist

Join our next KB Open Research Seminar on Tue, May 20, 3–4pm on the German Excellence Initiative. Prof. Heinze, Dr. Habicht & Dr. Tunger present their paper “Field size as a predictor of ‚excellence‘. The selection of subject fields in Germany’s Excellence Initiative”. Register: tinyurl.com/4te94k9r

The KB provides access to curated OpenAlex data on Zenodo: zenodo.org/records/1530... and Github: tinyurl.com/yc4w9vk4. Curations focus on GER research and comprise the following entities:
- Addresses
- Publishers
- Funding info
- Document types
- Transformative agreements
- Authors (in the future).

Our upcoming Open Research Seminars in May, June and July on the German Excellence Initiative, measuring disruption in science and evaluative bibliometrics in the DORA/CoARA context:

75% of US scientists who answered Nature poll consider leaving More than 1,600 readers answered our poll; many said they were looking for jobs in Europe and Canada.

75% of US scientists who answered Nature poll consider leaving www.nature.com/articles/d41...

Heatmaps for eight infections diseases in the US: poilio, mumps, rubbella. hepatitis A, ... Each one has the US states in rows and year in columns, with a thick black line showing the year that a vaccine against that disease was introduced. Cells are colored by the number of cases of that disease.

Graphic #rstats reminder that vaccines work 📊
www.databrewer.co/R/gallery/gg...

📊 #psy6135 #rstats Today's topic: Visualizing Uncertainty
Topics: Visualizing distributions, Error bars, Bayesian stuff, fitted curves, ...
Thx to @elibryan.bsky.social, @mjskay.com, @clauswilke.com, @cedricscherer.com and other folks I stole from.
💻 Slides-- friendly.github.io/6135/lecture...

Science has covered the paper in the In Brief section www.science.org/doi/10.1126/...

The 30th Nordic Workshop on Bibliometrics and Research Policy takes place in Bergen, Norway October 29-31 this year. The Call for Abstracts is now out, and submissions open March 17th. Presentations and posters alike are welcome. Hope to see you there?

6/ Bottom line: Direct and irresponsible use of scientometric indicators as performance measures may have unintended negative consequences that may undermine research integrity.

If you're interested in the full study, check it out here: link.springer.com/article/10.1...

5/ What kind of misconduct did we find?
In WoS/Scopus journals: data falsification, fake peer reviews, authorship manipulation.
In non-indexed journals: plagiarism and self-plagiarism were more common.
The type of misconduct seems to depend on the journal's visibility and standards.

4/ However, there was no significant difference in retractions from non-indexed journals between RUEI and non-RUEI universities. This shows that focus is on WoS/Scopus metrics.

3/ Our study shows that RUEI universities had more retracted papers in WoS/Scopus journals compared to non-RUEI universities. It seems the pressure to publish in these visible journals created incentives for questionable research practices and misconduct.

2/ The RUEI, also known as Project 5-100, aimed to boost global competitiveness by rewarding universities based on their publication counts in internationally indexed journals. 3 of 7 KPIs were related to the number of articles in journals indexed in WoS)/Scopus.

🚨 New paper alert! 🚨

1/ We investigated the link between research evaluation systems and research misconduct, focusing on the Russian University Excellence Initiative (RUEI) as an example. Here’s a short thread

Bibliometric databases coverage (March 2025) including the number of Open Access items (millions)

Progress in #ResearchAssessment in Germany: @dfg.de recommendation for a hybrid CV-template:
- no picture
- no birthdates
- no JIFs
- no h-index
- non-traditional pubs allowed

i.e. more substance, less highly abused metrics

Inspired by #CoARA @dorassessment.bsky.social

www.dfg.de/resource/blo...

On author self-citations as carriers of scientific topic structure information Abstract. Author self-citations are a somewhat controversial phenomenon. Some scholars maintain they are a normal, even indispensable, part of scientific referencing practice, while others claim they ...

A new publication from the KB: "On author self-citations as carriers of scientific topic structure information", take-away: clustering solution quality can be improved by emphasizing self-citation link weights. direct.mit.edu/qss/article/...

Nordic workshop in bibliometrics and research policy will be arranged in Bergen October 29-31. Submissions are welcome! www.nifu.no/nwb2025/

Citations drop after scientists accused of sexual, but not scientific, misconduct Findings contradict researchers’ own beliefs about how they would cite accused peers

Citations drop after scientists accused of sexual, but not scientific, misconduct | Science | AAAS www.science.org/content/arti...

Remember the old #Twitter and when you decided to migrate? How important were your peers for your decision?

In our current study, we use data from 250k researchers during the 2022 migration to #Mastodon to examine #socialinfluence within the research community.

👉 doi.org/10.23668/psy...

🧵1/4

KB Open Research Seminar
The influence of geopolitics on research activity and international collaboration in science: the case of Russia (& notes on the RIC indicator for measuring cooperation between countries) by Gunnar Sivertsen:
bibliometrie.info/wp-content/u...

Hello Bluesky, I guess I should introduce myself.
I'm Jack, I enjoy bibliometrics and Open Science research.
When I have time I'm also pursuing my PhD.
I created the Open Research Converter (the ORC) orc-demo.gesis.org, and looked at the reference coverage of OpenAlex arxiv.org/abs/2401.16359.

BibexPy: Harmonizing the bibliometric symphony of Scopus and Web of Science Bibliometric analyses frequently encounter issues such as duplicate records, missing metadata, and inconsistent formats, reducing reliability and effi…

"This paper presents a Python-based software solution that addresses these challenges by merging datasets from Scopus and Web of Science, performing DOI-based deduplication, and enhancing metadata using APIs such as Unpaywall and Semantic Scholar."
www.sciencedirect.com/science/arti...

📢 Call for Papers 📢The IJER Special Issue "Understanding Academic Freedom: Cross-Disciplinary, Cross-Regional, and Cross-National Perspectives" is open for submissions
Guest editors: Zoltán Rónay, Marcelo Marques, Daniela Craciun, Gergely Kováts
www.sciencedirect.com/special-issu...
#academicfreedom

Call for Proposals The theme for the CHER 2025 Annual Conference is “Adapting to Change: Higher Education Systems, Institutions and Academics in a Transforming World”. The political, social, and economic world has been ...

Dear #HigherEd #Research and #Policy community - you are welcome to come!

📢 37th #CHER 2025 Annual Conference

📢3-5 September 2025, Poznan

📢Abstract Submission Deadline: 7 March 2025

📢Call for Proposals cher2025.amu.edu.pl/call-for-pro...
✅Vienna 2023
✅Luxembourg 2024
✅Poznan 2025

RTplease!

GitHub - dbann/pubanalyser: Estimates how much an author has spent on publications, including the % of for profit publishers Estimates how much an author has spent on publications, including the % of for profit publishers - dbann/pubanalyser

github.com/dbann/pubana...