Most common uses of AI in Healthcare I review some data on usage of AI in healthcare, and conclude that the most common uses in 2025 are probably (A) giving personalised health information to patients and (B) helping clinicians write …

New blog: Most common uses of AI in Healthcare

Data on usage of AI in healthcare suggests that most common uses in 2025 are probably (A) giving personalised health information to patients and (B) helping clinicians write documents.

ehudreiter.com/2025/10/21/m...

One of my main goals for 2025-26 is to help my 6 senior PhD students submit their PhDs before I retire. Glad to say that Nicolay Babakov has now done so, with viva scheduled for Dec. Other five students seem to be on track, which is encouraging.

Somewhat frustrated yesterday to once again read ACL paper which did all sorts of complex things (including the usual results tables showing best approach) on garbage data. With minimal ack of this in limitations. Most fundamental rule of CS is Garbage In, Garbage Out

Good diagrams for research papers Ive seen a number of diagrams recently which are too complicated and difficult to understand. I explain some of the problems I see and give advice.

New blog: Good diagrams for research papers

Ive seen a number of diagrams recently which are too complicated and difficult to understand. I explain some of the problems I see and give advice.

ehudreiter.com/2025/10/08/g...

What Matters in a Measure? A Perspective from Large-Scale Search Evaluation | Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

Really interesting paper on real-world evaluation in IR. I should learn more about eval in IR, its not something Ive ever properly looked at
dl.acm.org/doi/10.1145/...

Several people have asked me recently if I will still be able to contribute to research projects after I retire in summer 2026. Absolutely! I will have emeritus statius, and am very hapy to remain involved in research projects at Aberdeen amd elsewhere.

Aberdeen CS is hiring! We are especially interested in hiring new faculty in NLP. Closing date is 8 Oct. For more info, see below (or contact me)

www.abdn.ac.uk/jobs/vacanci...

Reflections on blogging I am often asked about my experience blogging, sometimes by people who are considering writing their own blog. In this “meta” blog, I summarise my thoughts and experiences about my blog…

New blog: Reflections on blogging

I am often asked about my experience blogging, sometimes by people who are considering writing their own blog. In this “meta” blog, I summarise my thoughts and experiences about my blog.

ehudreiter.com/2025/09/23/r...

Aberdeen CS will probably be looking for a new lecturer in NLP. Formal advert is not out yet, but feel free to contact me informally if interested.

Picture of the One Pillar Pagoda in Hanoi, a pagoda raised up over a green pond surrounded by greenery

The registration page for #INLG2025 is now live! Join us in Vietnam at the Oct 29 - Nov 2 for the best conference on #NaturalLanguageGeneration

2025.inlgmeeting.org/registration...

Curious to see what will be presented? Check out this list of accepted papers! 2025.inlgmeeting.org/accepted-pap...

Defining hallucination is not straightforward Most academic work assumes that hallucination is a binary feature: either something is a hallucination or it is not a hallucination. But this is too simplistic. In real-world contexts we see many s…

New blog: Defining hallucination is not straightforward

Many researchers assume that hallucination is a binary feature; either something is a hallucination or it is not. This is too simplistic. I describe some of the issues I have seen below.

ehudreiter.com/2025/09/10/d...

At ACL, I engaged with 50 papers (went to oral, talked to poster person). Decided (looked at paper sometimes), that 3 of these robust, interesting, relevant to me; 2 of these 3 won awards. Hum, maybe in future I should focus on 40 award papers, ignore the other 3000?

Encouraging safer driving with NLG apps I am very excited by recent positive evaluations of NLG apps developed by my students to encourage safer driving in UK and Nigeria. We see statistically significant reductions in unsafe driving inc…

Excited by recent positive evaluations of NLG apps developed by my students to encourage safer driving in UK and Nigeria. We see stat sig reductions in unsafe driving incidents in both countries.

ehudreiter.com/2025/09/03/e...

Last week I had to deal with two cases of papers containing hallucinated references. This is not acceptable! Shows complete disdain for understand prev work, and suggests rest of paper may be fabricated.

Ok to use LLM to suggest related work, but read (or at least skim) them!

Watched recording of ACL panel on generalisability (recommended to me). I share concerns about "LLM popcorn", but my biggest concern about NLP is lack of research diversity. Everyone does LLM, few people do impact or qual eval, little interest in genuine collab with other fields

I hate pay-to-publish The academic world has changed in many ways since I got my PhD in 1990. One of the worst changes is that researchers in 2025 usually need to pay thousands of pounds to publish their work. This is u…

New blog: I hate pay-to-publish

The academic world has changed since I got my PhD in 1990. One of the worst changes is that researchers now often pay thousands of pounds to publish their work. Unfair to researchers with limited funding, and bad for science.

ehudreiter.com/2025/08/19/i...

When combinations of humans and AI are useful: A systematic review and meta-analysis - Nature Human Behaviour Vaccaro et al. present a systematic review and meta-analysis of the performance of human–AI combinations, finding that on average, human–AI combinations performed significantly worse than the best of ...

Very interesting meta-analysis of human-AI collab. Shows more effective in content creation (eg report writing) than in decision making, which does not surprise me

When combinations of humans and AI are useful: A systematic review and meta-analysis

www.nature.com/articles/s41...

Excited to announce the first-ever Workshop for Young Researchers in Natural Language Generation (YNLG), supported by @siggen.bsky.social, taking place on October 29, 2025 in Hanoi, Vietnam, co-located with INLG 2025.
Call for Submissions is out now!

ynlg-workshop.github.io

More on evaluating impact I recently published a paper and gave a talk about evaluating real-world impact. I got some great feedback from this, and summarise some of the suggested papers (including more examples of impact e…

New blog: More on evaluating impact

I got great feedback from recent paper and talk on eval impact, and summarise some of the suggested papers (including more examples of impact eval) and insightful comments (eg, about eval “ecosystem”) I received.

ehudreiter.com/2025/08/05/m...

I'll be at ACL next week (Tue-Thur, not Sun/Mon). Look forward to meeting old friends and new people who want to connect! Ill also be giving an invited talk on impact evaluation at the GEM workshop on Thur 31 July

Really happy that this survey of NLP in cancer care, from my student Mengxuan Sun , has finally appeared (its been a saga). One key but depressing finding is that evaluation quality is uniformly dreadful by medical standards; NLP researchers just dont seem to care...

doi.org/10.1016/j.ar...

Motivated by recent discussion with my group:
Ignore subjective statements such as "I find LLMs to be incredibly useful for XX", especially when made by people (such as AI companies or gurus) who have strong biases/incentives/COI .

Nice example of using RCT to measure real-world impact of LLMs (and discovering that it is disappointing)

Good point, in some cases I have struggled to convince companies to publish. But in other cases we could publish. I guess depends on the company and the people who make this decision, and also on what is being published (eg very hard to publish negative result about company's product!)

I'll also give an invited talk about impact evaluation at the ACL GEM workshop

We Should Evaluate Real-World Impact The ACL community has very little interest in evaluating the real-world impact of NLP systems. A structured survey of the ACL Anthology shows that perhaps 0.1% of its papers contain such evaluations; ...

Ive written a "Last Word" opinion piece for CL about evaluating real-world impact. It
* looks at how impact can be evaluated
* shows via a structured survey that perhaps 0.1% of ACL Anth papers measure real-world impact
* discusses why this is the case

arxiv.org/abs/2507.05973

Looked at Google Scholar, nice to see that my h-index has reached 60

Patients want to know what information an AI model considers My student Adarsa Sivaprasad is looking into what questions users of an AI prediction model actually have, and how these should be answered. Amongst other things, users seem to have more questions …

If a woman is considering IVF and uses an AI model to predict liklihood of success (having a baby), what explanations would help her trust the model? Info about what information the model considers is more important than info about how model works

ehudreiter.com/2025/06/25/p...

Congratulations to my student Adarsa Sivaprasad for winning a best PhD student poster award at Healtac , for her work on "A conversational agent to address patient needs for out-of-distribution explanations"!

Fascinating keynote by Alison ONeil at Healtac. One key point is that in her space, omissions are bigger problem than hallucination since harder for readers to detect. Not surprisingly omissions more common for rare phenomenon