Steffen Herbold

Very interesting paper that shows that LLMs are good. But not good enough. I wish the following from the conclusion would have made it into the abstract:

😂

Just accepted at TMLR:

We found evidence of copyright violations by LLMs even when we ask questions that were not part of the training. Indeed, we found that the amount of memorized content was independent from the questions being part of the training or not.

openreview.net/forum?id=ddo...

Why language models hallucinate OpenAI’s new research explains why language models hallucinate. The findings show how improved evaluations can enhance AI reliability, honesty, and safety.

This just in: Leading AI firm discovers confidence thresholds. More on this exciting development in news at 11.

openai.com/index/why-la...

(Honestly, OpenAI!?)

Scientific impact and achievement, redefined:
Huge congrats to #Fraunhofer IIS on winning an #Emmy for their JPEG XS compression standard 🏆🎉 […]

re

(I miss IRC)

(Now I feel old)

Dear all,

please enjoy your complementary "European Professor goes on Holiday" message.

See you in September.

Yours sincerely,
A European Professor

Good news (for me!) my gender bias paper from 2023 still replicates with GPT-5.
Bad news (for everyone!) my gender bias paper from 2023 still replicates with GPT-5.
arxiv.org/pdf/2308.14921
hkotek.com/blog/gender-...

I wonder what my PhD students will think, once they discover that "someone" glued the three laws to the wall in the hallway. 🙃

Newton's Laws of Graduation, Part 2 - The Second Law

Newton's Laws of Graduation, Part 3 - The Third Law 😆

Success, a luxury problem, and its solution:
🎉 Our quiz is a huge success and incredibly popular on YouTube with now over 100,000 views.
😐 We cannot answer all the feedback and comments individually anymore.
😀 We write a follow up article to answer the most important questions.

Partial Colexifications Improve Concept Embeddings Arne Rubehn, Johann-Mattis List. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

It is official, our two long papers at #ACL2025 have now been published. Common work with Arne Rubehn (Concept Embeddings), and Frederic Blum and @sherbold.bsky.social (Automated Language Affiliation).

aclanthology.org/2025.acl-lon...
aclanthology.org/2025.acl-lon...

My debut as TV-Show moderator - now live on Youtube.

We had a lot of fun with how the five professors answered questions on topics ranging from 90's music, counting peas, size of Asian countries, etc.

The only drawback: it is only available in German.

P.S. The humans won.

YouTube video by Universität Passau Quizshow 5 gegen KI - Professorenteam tritt gegen KI an

Wie schlägt sich KI gegen professorale Expertise? Die Quiz-Show unter Moderation von @sherbold.bsky.social ist nun in voller Länge online. Wer sich vorab selbst mit der KI messen möchte, kann dies per Online-Quiz tun: www.digital.uni-passau.de/beitraege/20...

#KeepCALLM #5gegenKI

That was so much fun. I look forward to the video 😃

I'll just leave that quote here ...

Yesterday: Let's try to ground AI models in reality.
Now: Let's try to ground reality on AI models.

Fixes a lot of issues. I am impressed. 😅

They should call it LLM as a Physicists, then it gets accepted by the community ... right? (Looking at you, everybody trusting LLM as a judge!)

Happy to share that we published MAMUT @tmlrorg.bsky.social. We defined multiple data augmentation approaches to get more diverse mathematical data and show this improves pre-training.

Congrats to my student Jonathan Drechsel for his first publication! 🎉

www.fim.uni-passau.de/en/ai-engine...

No need, I can already feel the @icseconf.bsky.social paper bidding approaching 🙃

Starting the weekend on a Friday at 4pm with an empty inbox feels kind of strange. Good, but strange.

How much energy is needed to generate an image? 🎨🧠⚡️
Up to 4.08 Wh — like charging your phone to 40%!
In our new study we tested 17 models & 9,000+ runs.

Other key finds:
⚡️ Model energy use varies up to 46x
📐 Resolution matters, prompts don't
🛠️ Quantization ≠ savings

📄 Preprint: lnkd.in/dKWWAETW

YouTube video by Universität Passau Ringvorlesung - Sprachmodelle: ein neuer Zugang zum Recht?

Können #LLMs einen neuen Zugang zum Recht eröffnen? Darüber spricht Brian Valerius, Professor für #KI im #Strafrecht, mit Rechtsanwalt Sven Galla, der KI bereits in der Praxis einsetzt.

📅 Donnerstag, 10. Juli, 18 Uhr
📍 Hörsaal 13

Mehr Infos: www.digital.uni-passau.de/generative-s...

#KeepCALLM

The truly impressive thing about Zoom is that whenever they update the UI, it gets worse.

Evaluating Large Language Models on Non-Code Software Engineering Tasks Large Language Models (LLMs) have demonstrated remarkable capabilities in code understanding and generation; however, their effectiveness on non-code Software Engineering (SE) tasks remains underexplo...

New pre-print: If you are wondering which models are good for non-code software engineering tasks, take a look at this work from my student Fabian Pena.

Also: Look at it if you want to know how to use Bayesian stats for ranking models.

arxiv.org/abs/2506.10833

Reviews so far this year: 11 journal papers, 8 conference papers, and 6 registered report protocols.

And I already feel like I decline almost all incoming requests...

Congrats especially also to our main author Frederic Blum!

Wow, our ACL paper with @lingulist.de is selected as oral presentation at the ACL main conference - less than 10 percent of the accepted papers get this honor 🤯

Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task This study explores the neural and behavioral consequences of LLM-assisted essay writing. Participants were divided into three groups: LLM, Search Engine, and Brain-only (no tools). Each completed thr...

❝Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels. These results raise concerns about the long-term educational implications of LLM reliance and underscore the need for deeper inquiry into AI's role in learning.❞ arxiv.org/abs/2506.08872

Ars Technica News and reviews, covering IT, AI, science, space, health, gaming, cybersecurity, tech policy, computers, mobile devices, and operating systems.

The big lawsuit from Disney and Co finally has arrived. This will be interesting: arstechnica.com/ai/2025/06/i...