Berk Ustun

Berk Ustun

@berkustun.bsky.social

Assistant Prof at UCSD. I work on safety, interpretability, and fairness in machine learning. www.berkustun.com

2,644 Followers 454 Following 43 Posts Joined Sep 2023
2 months ago
6.005: Software Construction

Honestly most topics in this Software Engineering course are still very relevant:

ocw.mit.edu/ans7870/6/6....

Things I've used in the past few months:

- Writing Good Specs

- Safety Mechanisms (Tests, Assertions, Immutability etc.)

- Design Patterns (GUIs, Parallelism, Parser Generators)

1 0 0 0
3 months ago

The easiest / cleanest solution would be free registration at the conference.

1 0 0 0
3 months ago
Post image

I'll be @neuripsconf.bsky.social presenting Strategic Hypothesis Testing (spotlight!)

tldr: Many high-stakes decisions (e.g., drug approval) rely on p-values, but people submitting evidence respond strategically even w/o p-hacking. Can we characterize this behavior & how policy shapes it?

1/n

17 4 1 0
3 months ago

Spread the word! 📢 The FATE (Fairness, Accountability, Transparency, and Ethics) group at @msftresearch.bsky.social in NYC is hiring interns and postdocs to start in summer 2026! 🎉

Apply by *December 15* for full consideration.

51 29 1 0
4 months ago
Preview
HMRC trial of child benefit crackdown wrongly suspected fraud in 46% of cases Exclusive: Almost half of families flagged as emigrants based on Home Office travel data were still living in UK

UK government project using AI to find benefit fraud resulted in:

- A 46% false fraud rate
- Anguish for families who were wrongly accused of fraud and had benefits stopped
- Months of additional work for government, setting up a hotline, correcting false fraud

www.theguardian.com/society/2025...

139 92 12 17
4 months ago

I’m giving an IDE seminar at @mitsloan.bsky.social tomorrow at 11am, on optimizing AI as decision support. Joint work w/ @ziyang.bsky.social @yifanwu.bsky.social @jasonhartline.bsky.social @berkustun.bsky.social
Come by if you’re around!

www.eventbrite.com/e/fall-2025-...

12 4 0 0
5 months ago

Who teaches an undergraduate principles of programming languages class? Looking for some inspiration to teach one at UCSD

6 1 2 0
6 months ago
Preview
The Actuary's Final Word on Algorithmic Decision Making Paul Meehl's foundational work "Clinical versus Statistical Prediction," provided early theoretical justification and empirical evidence of the superiority of statistical methods over clinical judgmen...

In a new paper, I try to resolve the counterintuitive evidence of Meehl’s “clinical vs statistical prediction” problems: Statistics only wins because the game is rigged.

29 9 3 0
7 months ago

Time for XAI for Code? 🙃

1 0 0 0
7 months ago
Post image

Machine learning models can assign fixed predictions that preclude individuals from changing their outcome. Think credit applicants that can never get a loan approved, or young patients that can never get an organ transplant - no matter how sick they are!

1 1 1 0
7 months ago
Preview
Understanding Fixed Predictions via Confined Regions Machine learning models can assign fixed predictions that preclude individuals from changing their outcome. Existing approaches to audit fixed predictions do so on a pointwise basis, which requires ac...

Excited to be chatting about our new paper "Understanding Fixed Predictions via Confined Regions" (joint work with @berkustun.bsky.social, Lily Weng, and Madeleine Udell) at #ICML2025!

🕐 Wed 16 Jul 4:30 p.m. PDT — 7 p.m. PDT
📍East Exhibition Hall A-B #E-1104
🔗 arxiv.org/abs/2502.16380

5 3 1 0
8 months ago
Preview
Explanations are a means to an end Modern methods for explainable machine learning are designed to describe how models map inputs to outputs--without deep consideration of how these explanations will be used in practice. This paper arg...

Paper: www.arxiv.org/abs/2506.22740

Blog post: statmodeling.stat.columbia.edu/2025/07/02/w...

23 3 2 1
8 months ago

ExplainableAI has long frustrated me by lacking a clear theory of what an explanation should do. Improve use of a model for what? How? Given a task what's max effect explanation could have? It's complicated bc most methods are functions of features & prediction but not true state being predicted 1/

46 8 2 0
8 months ago
screenshot of title and authors (Jakob Schoeffer, Maria De-Arteaga, Jonathan Elmer)

Having a lot of FOMO not being able to be in person at #FAccT2025 but enjoying the virtual transmission 💻. Tomorrow Jakob will be presenting our paper "Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest".

13 1 1 0
8 months ago

Explanations don't help us detect algorithmic discrimination. Even when users are trained. Even when we control their beliefs. Even under ideal conditions... 👇

11 0 0 1
9 months ago
Post image

*wrapfig entered the document*

5 1 1 0
10 months ago
Post image

“Science is a smart, low cost investment. The costs of not investing in it are higher than the risk of doing so… talk to people about science.” - @kevinochsner.bsky.social makes his case to the field #sans2025

170 55 7 7
10 months ago

I tried to be nice but then they said that saying please and thanks costs millions.

4 0 0 0
10 months ago

Hey AI folks - stop using SHAP! It won't help you debug [1], won't catch discrimination [2], and makes no sense for feature importance [3].

Plus - as we show - it also won't give recourse.

In a paper at #ICLR we introduce feature responsiveness scores... 1/

arxiv.org/pdf/2410.22598

29 8 3 0
10 months ago

When RAG systems hallucinate, is the LLM misusing available information or is the retrieved context insufficient? In our #ICLR2025 paper, we introduce "sufficient context" to disentangle these failure modes. Work w Jianyi Zhang, Chun-Sung Ferng, Da-Cheng Juan, Ankur Taly, @cyroid.bsky.social

11 5 1 0
10 months ago
Post image

Denied a loan, an interview, or an insurance claim by machine learning models? You may be entitled to a list of reasons.

In our latest w @anniewernerfelt.bsky.social @berkustun.bsky.social @friedler.net, we show how existing explanation frameworks fail and present an alternative for recourse

16 7 1 1
10 months ago

Absolute banger.

30 4 4 1
10 months ago
Post image

Many ML models predict labels that don’t reflect what we care about, e.g.:
– Diagnoses from unreliable tests
– Outcomes from noisy electronic health records

In a new paper w/@berkustun, we study how this subjects individuals to a lottery of mistakes.
Paper: bit.ly/3Y673uZ
🧵👇

12 2 1 0
10 months ago
Preview
Learning under Temporal Label Noise Many time series classification tasks, where labels vary over time, are affected by label noise that also varies over time. Such noise can cause label quality to improve, worsen, or periodically chang...

🚨 Excited to announce a new paper accepted at #ICLR2025 in Singapore!

“Learning Under Temporal Label Noise”

We tackle a new challenge in time series ML: label noise that changes over time 🧵👇

arxiv.org/abs/2402.04398

5 1 1 1
1 year ago

is this a rhetorical question?

5 0 0 0
1 year ago
Post image
1 0 1 0
1 year ago
Preview
Home | HCXAI ACM CHI 2025 Workshop on Human-Centered Explainable AI (HCXAI). May 2025 (Yokohama, Japan & hybrid). Submit your Paper (EasyChair)

The CHI Human-Centered Explainable AI Workshop is back!

Paper submissions: Feb 20

hcxai.jimdosite.com

26 12 0 0
1 year ago
Post image

🧵on the CFPB and less discriminatory algorithms.

last week, in its supervisory highlights, the Bureau offered a range of impressive new details on how financial institutions should be searching for less discriminatory algorithms.

16 5 1 0
1 year ago
Post image

Also

0 0 0 0
1 year ago
Post image


Engaging discussions on the future of #AI in #healthcare at this week's ICHPS, hosted by @amstatnews.bsky.social.

JCHI's @kdpsingh.bsky.social shared insights on the safety & equity of #MachineLearning algorithms and examined bias in large language models.

9 4 1 0