Constanza Fierro

🧠🤖 Join us for the launch of the Center for Philosophy of AI (University of Copenhagen)!

📅 Sept 3, 13:00-17:00
📍CPH Conference

Keynotes on philosophy of LLMs by @parismarx.com, Ellie Pavlick, @dcm.social.sunet.se.ap.brid.gy, Tom Sterkenburg & @zhijingjin.bsky.social

Register: cpai.ku.dk

🤔Do Vision and Language Models Share Concepts? 🚀
We present an empirical evaluation and find that language models partially converge towards representations isomorphic to those of vision models. #EMNLP

📃 direct.mit.edu/tacl/article...

🙋🏽‍♀️🙋🏽‍♀️

6/🧵 We hope that the connection we provide to epistemology can inform and motivate better evaluations and claims regarding knowledge in LLMs 🚀

5/🧵 We also ask philosophers and computer scientists for their opinion on this matter: can we even say that an LLM knows a fact? We find some intriguing discrepancies between the two groups 🌗

4/🧵Using the definitions, we identify gaps in how current NLP research conceptualizes knowledge compared to epistemology 🔍. We also highlight open questions and new research opportunities to improve knowledge evaluations.

3/🧵Philosophers have thought and discussed about “knowledge” for a long time. So we look to epistemological research for definitions of knowledge 🧐 and interpret them in the context of LLMs 📝

2/🧵 However, inconsistencies may arise with paraphrases or with logically related facts 💥. Would it really know that “Lionel Messi plays for Inter Miami” if it fails to predict where Lionel Messi resides? ⚽️

1/🧵 LLMs seem to know a lot of facts about the world, GPT-4 correctly completes “The capital of Germany is ___” with Berlin, so it must know that, right?

Heading to Miami! To present 👇🌴

Does GPT-4 truly “know” that the Earth is round? 🤔

First, some definitions 💡

“Defining Knowledge: Bridging Epistemology and Large Language Models”

📄 arxiv.org/abs/2410.02499
🗣️ #EMNLP2024

Hey Maria! Thanks for the list, could you add me? 😀