Akari Asai

Forbes 30 Under 30 2025: Healthcare & Science Discovering new worlds, in our cells and outer space.

Honored to be named to the Forbes 30 Under 30 Asia 2025 in Science!
Grateful for the recognition of my Ph.D. work on Retrieval-Augmented LMs, and excited to keep pushing the boundaries of reliable and efficient language models.
🔗 forbes.com/30-under-30/...
More updates soon… 👀

Sad to miss #ICLR2025 this year, but my amazing co-authors will be there in person to present Pangea!
neulab.github.io/Pangea/
I’ll be at the Foundation Models for Science conference at Simons Foundation, NYC next week, then heading to NAACL (more details soon).
Let’s catch up if you’re around!✨

Real user queries often look different from the clean, concise ones in academic benchmarks - ambiguity, full of typos, and much less readable.
We show that even strong RAG systems quickly break under these conditions.
Awesome project led by
@neelbhandari.bsky.social and @tianyucao.bsky.social!!

31% of US adults use generative AI for healthcare 🤯But most AI systems answer questions assertively—even when they don’t have the necessary context. Introducing #MediQ a framework that enables LLMs to recognize uncertainty🤔and ask the right questions❓when info is missing: 🧵

CopyBench (EMNLP 2024, led by @tomchen0112.bsky.social)
Oral at regulatableml.github.io & Poster at redteaming-gen-ai.github.io

tldr: We benchmarked LLMs' literal/non-literal copying of copyrighted content—risks found even in 8B models.

Detais: www.arxiv.org/abs/2407.07087

MassiveDS (led by @rulinshao.bsky.social) Wednesday Poster at 11-2 pm at West Ballroom#7203

TLDR: We demonstrated scaling retrieval corpora of Retrieval-Augmented LMs to 1.4T helps & achieves more compute-optimal scaling

Details: retrievalscaling.github.io

Excited to attend #NeurIPS2024 in person! I’ll be presenting MassiveDS and CopyBench. Details below 🧵👇

Let’s catch up and chat about:
- LLMs & Retrieval-Augmented/Augmented LMs
- LLM Applications for science (e.g., OpenScholar) & others
- Ph.D./faculty apps
...and more!

Oh that's a screenshot of my website. Here's link to my CV akariasai.github.io/assets/pdf/a...

I would love to hear about any opportunities that might be a good fit!! You can find my contact info and CV on my website. akariasai.github.io. I am attending NeurIPS in person so let’s chat!

🏆 Recognition & Impact: My work has earned EECS Rising Stars 2022, the MIT Tech Review Innovator Award (Japan 2024), paper awards at ACL & NeurIPS, and the IBM Fellowship. My work has been featured in medias like MIT Tech Review, Forbes and VentureBeat.

Ai2 OpenScholar: Scientific literature synthesis with retrieval-augmented language models | Ai2 Ai2’s & UW’s OpenScholar, a retrieval-augmented LM, helps scientists navigate and synthesize scientific literature.

🌍 Making Real-World Impacts
Retrieval-Augmented LMs tackle critical challenges like:
1️⃣ Unreliable LMs in expert domains
2️⃣ Information access inequity across languages
I launched OpenScholar for scientific synthesis—20k+ demo requests in week 1! Details: allenai.org/blog/opensch...

🛠 Building the Foundations:
Retrieval-augmented LMs need more than off-the-shelf models. I developed advanced training/inference algorithms & architectures, including Self-RAG (ICLR 2024 Oral; NeurIPS Workshop Hon. Mention) for adaptive retrieval & self-critique.
Learn more:
selfrag.github.io

When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, Hannaneh Hajishirzi. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

🔍 Establishing the Necessity
My work showed that scaling LLMs alone doesn’t solve issues like hallucinations or obsolete knowledge and is compute suboptimal, and Retrieval-Augmented LMs address these challenges. See our ACL 2023 Best Video Award paper:
aclanthology.org/2023.acl-lon...

I’m on the academic job market this year! I’m completing my @uwcse.bsky.social @uwnlp.bsky.social Ph.D. (2025), focusing on overcoming LLM limitations like hallucinations, by building new LMs.
My Ph.D. work focuses on Retrieval-Augmented LMs to create more reliable AI systems 🧵

congrats @akariasai.bsky.social:

🔬 retrieval augmented LM for science literature
🧬 open data, weights, index, code, etc
⚗️ new eval suite for science literature tasks
🔭 demo to play w the model

encourage checking out to see what scientific LMs can/cant do today w open research artifacts

Super exciting RAG prototype @akariasai.bsky.social build on top of Semantic Scholar!

I love how it returns competent research answers for seemingly out CS domain questions, eg “what’s a bell?” openscholar.allen.ai/query/69cf13...

it’s good in domain too 😉

A photo of Boulder, Colorado, shot from above the university campus and looking toward the Flatirons.

I'm recruiting 1-2 PhD students to work with me at the University of Colorado Boulder! Looking for creative students with interests in #NLP and #CulturalAnalytics.

Boulder is a lovely college town 30 minutes from Denver and 1 hour from Rocky Mountain National Park 😎

Apply by December 15th!

8/ ❤️Acknowledgements:
OpenScholar is the result of a collaborative effort UW, Ai2 and many others!
Huge thanks to our incredible team including experts from CS, Bio, and physics, for making this possible!
We’d love your feedback! Reply or email us with questions, ideas, or use cases✨

8/ 🧪 Summary
Try it out: openscholar.allen.ai
Read more: allenai.org/blog/opensch... – we discuss more details as well as limitations of OpenScholar, based on our beta testing with CS researchers!
Code & data: github.com/AkariAsai/Op...
Paper: openscholar.allen.ai/paper

7/ 🌐 What’s next?
We're just getting started with OpenScholar! 🚀
Expanding domains: Support for non-CS fields is coming soon. Public API: Full-text search over 45M+ papers will be available shortly.
Try the OpenScholar demo and share your feedback!
openscholar.allen.ai

6/ 💾 Open Access [2]:
📂 OpenScholar Datastore (45M+ papers up to 2024/10): huggingface.co/datasets/Ope...
📊 ScholarQABench: github.com/AkariAsai/Sc...
👩‍🔬 Human evaluation interface: github.com/AkariAsai/Op...

6/ 💾 Open Access [1]:
Prior work in this area has relied on proprietary LMs and/or released only a subset of datastore
We're releasing
Demo: openscholar.allen.ai
🔓 Code & model checkpoints:
github.com/AkariAsai/Op...
huggingface.co/collections/...

5/ 📊 Exert Evaluation Results:
We further conduct expert evaluations with scientists across CS, Bio and Physics, comparing OS against expert answers.
Scientists preferred OpenScholar-8B outputs compared to human-written answers in majority of the times, thanks to its coverage

5/ 📊 Automatic Results:
So how good OpenScholar?
On ScholarBench, OpenScholar-8B surpassed GPT-4o, concurrent PaperQA2, and other models in factuality & citation accuracy despite being many times cheaper!

4/ 🧪New dataset: ScholarBench
A benchmark for evaluating scientific language models on real-world, open-ended questions requiring synthesis across multiple papers. 🌟
📚 7 datasets across four scientific disciplines
🧑‍🔬 2,000+ expert-annotated question and 200 answers
📊 Automated metrics

3/ 🔍 What is OpenScholar?
It's a retrieval-augmented LM with
1️⃣ a datastore of 45M+ open-access papers
2️⃣ a specialized retriever and reranker to search the datastore
3️⃣ an 8B Llama fine-tuned LM trained on high-quality synthetic data
4️⃣ a self-feedback generation pipeline

2/ 🏛️ On the shoulders of giants
With millions of papers published yearly, keeping up with scientific literature has become a monumental challenge. ᴏᴘᴇɴꜱᴄʜᴏʟᴀʀ aims to help researchers navigate this vast landscape by synthesizing grounded, citation-supported answers from academic papers.

1/ Introducing ᴏᴘᴇɴꜱᴄʜᴏʟᴀʀ: a retrieval-augmented LM to help scientists synthesize knowledge 📚
@uwnlp.bsky.social & Ai2
With open models & 45M-paper datastores, it outperforms proprietary systems & match human experts.
Try out our demo!
openscholar.allen.ai