Denied a loan, an interview, or an insurance claim by machine learning models? You may be entitled to a list of reasons.
In our latest w @anniewernerfelt.bsky.social @berkustun.bsky.social @friedler.net, we show how existing explanation frameworks fail and present an alternative for recourse
I couldn't make it to ICLR this year but co-author @cyroid.bsky.social will be around to chat!
📄 Paper (ICLR ’25): arxiv.org/abs/2411.06037
💻 Key Findings & Prompts: github.com/hljoren/suff...
#RAG #ICLR2025
Our work suggests that solving RAG hallucination problems requires moving beyond just improving retrieval—we need models that can accurately determine when retrieved information suffices for answering and abstain when appropriate confidence thresholds aren't met.
Building on these insights, we developed a selective generation framework using both sufficient context signals and model confidence to decide when to respond vs. abstain—improving accuracy of responses by 2-10% for Gemini, GPT, and Gemma.
Intriguingly, models sometimes generate correct answers despite insufficient context. We taxonomize these cases: parametric knowledge bridging information gaps, yes/no questions with 50% chance of correctness, and instances where the context provides partial reasoning paths.
We analyzed standard QA datasets through our sufficient context lens and found a surprising percentage lack sufficient information: ~56% for Musique, ~56% for HotpotQA, and ~23% for FreshQA. This highlights the magnitude of the information retrieval challenge.
Conversely, smaller models (Mistral 3, Gemma 2) struggle even with sufficient context—either hallucinating or failing to extract answers from the provided information. Neither approach solves the fundamental RAG reliability challenge.
A major finding: When context is sufficient, larger models (Gemini 1.5 Pro, GPT-4o, Claude 3.5) excel. But when it's insufficient, they're more likely to hallucinate than abstain—presenting incorrect answers with high confidence.
When RAG systems hallucinate, is the LLM misusing available information or is the retrieved context insufficient? In our #ICLR2025 paper, we introduce "sufficient context" to disentangle these failure modes. Work w Jianyi Zhang, Chun-Sung Ferng, Da-Cheng Juan, Ankur Taly, @cyroid.bsky.social
