Jens Rupprecht's Avatar

Jens Rupprecht

@jrupprec.bsky.social

PhD student @ Chair for Data Science in the Economic and Social Sciences (University of Mannheim) Website: https://jrupprec.github.io/

39 Followers  |  48 Following  |  1 Posts  |  Joined: 13.05.2025  |  1.8674

Latest posts by jrupprec.bsky.social on Bluesky

Large Language Models for Social Research: Potentials and Challenges
YouTube video by MZES Methods Bites Large Language Models for Social Research: Potentials and Challenges

🚨 Content alert 🚨

New recording and workshop materials published!

➑️ Large Language Models for Social Research: Potentials and Challenges
πŸ‘€ @indiiigo.bsky.social (University of Mannheim)

πŸ“Ί youtu.be/p5wPJHK-74M
πŸ—’οΈ github.com/SocialScienc...

06.11.2025 14:20 β€” πŸ‘ 9    πŸ” 3    πŸ’¬ 0    πŸ“Œ 0

πŸ‘‹ #ACL2025NLP πŸ‡¦πŸ‡Ή @marlutz.bsky.social and I are presenting our poster on demographic representativeness of LLMs today!

πŸ•¦ 10:30-12:00
πŸ“ Hall X5 (board 1 or 14 according to different sources 🧐)

Here’s the paper on ACL anthology: aclanthology.org/2025.finding...

Drop by!

29.07.2025 07:31 β€” πŸ‘ 22    πŸ” 7    πŸ’¬ 0    πŸ“Œ 1
Post image

Chair for Data Science in the Economic and Social Sciences at University of Mannheim having lots of fun at #ic2s2 @janajung.bsky.social @wanlo.bsky.social @indiiigo.bsky.social @jrupprec.bsky.social @maximiliankreutner.bsky.social and Stefano Balietti

23.07.2025 15:22 β€” πŸ‘ 21    πŸ” 4    πŸ’¬ 0    πŸ“Œ 0

Really excited to also present this work at #IC2S2 next week in NorrkΓΆping! πŸŽ‰ I'd love to discuss how to produce LLM survey responses at my poster on Wed at 13:30 (Poster Session 2, Poster ID 68) πŸ“Š

18.07.2025 15:19 β€” πŸ‘ 18    πŸ” 6    πŸ’¬ 0    πŸ“Œ 0
A research setup for the evaluation of Answer Production Methods for closed-ended survey responses from LLMs. An LLM is prompted with a survey and an optional instruction, before a Answer Production Method is applied. These methods range from token-probabilities to open-ended text generation + classification. I then evaluated them against human survey answers and calculate individual-level accuracy as well as distribution alignment for sub-populations.

A research setup for the evaluation of Answer Production Methods for closed-ended survey responses from LLMs. An LLM is prompted with a survey and an optional instruction, before a Answer Production Method is applied. These methods range from token-probabilities to open-ended text generation + classification. I then evaluated them against human survey answers and calculate individual-level accuracy as well as distribution alignment for sub-populations.

LLMs are trained to produce open-ended responses πŸ“, but most survey items require closed-ended responses instead πŸ“Š

This Wed 11:00–12:30 at #ESRA25, I'll discuss the large impact that Answer Production Methods have on prediction results + share recommendations for methods and parameters. πŸ‘ˆ

14.07.2025 13:34 β€” πŸ‘ 10    πŸ” 4    πŸ’¬ 0    πŸ“Œ 2

Very excited to head to #IC2S2 next week! πŸŽ‰

In our project, we tested whether a psychological assessment can measure sexism in LLMs, and found that applying such tools to LLMs is not as straightforward as it seems.

Find me and my poster at Poster Session 1 (Tue 12:30-14:30) β€” hope to see you there

17.07.2025 19:14 β€” πŸ‘ 9    πŸ” 4    πŸ’¬ 0    πŸ“Œ 0
Post image

LLMs can generate synthetic survey responses, e.g. for imputation, but how reliable are they? πŸ“‹

At #IC2S2, I'll be sharing our research on the robustness of AI-generated responses to perturbations and if they mirror human survey biases. πŸ€–
Come by my poster on Tuesday between 1:30 and 3:30 p.m.

18.07.2025 14:16 β€” πŸ‘ 7    πŸ” 3    πŸ’¬ 0    πŸ“Œ 0

@jrupprec is following 20 prominent accounts