Kyle Lo

yess!! sry bout the x-axis, still thinkin how to make figure clearer

it's exactly what you're saying -- each point refers to a stage of development. our release has data+ckpts+evals for all stages we use (figure) and wanted to show how it compares to other models which typically only few stages

Research Internship, OLMo Seattle, WA

We're hiring too!

Olmo 3 was our biggest effort yet, but we're still a small team (67 authors!) compared to a lot of the big labs, which means everyone (especially interns) gets to own a major piece of the Olmo puzzle

job-boards.greenhouse.io/thealleninst...

🍏Try the model: playground.allenai.org
🍍Download the collection: huggingface.co/collections/...
🍌Read the blog: allenai.org/blog/olmo3
🍎And our 100+ page paper lol www.datocms-assets.com/64837/176364...

🍕Finally, we all know midtraining is an exciting time to get a ton of performance boost

But team organization to sustain consistent model improvements (without burnout) is important!

We have explorers "own" target capabilities & centralized assessment team run "integration tests"

🍨Data quality signals matter but also how you use them!

Traditional ways of using data quality is to threshold: Define a cutoff and take all the documents above that threshold.

But why not sample *proportional* to data quality?

We use Quality-Aware Upsampling to do exactly this

🍣Data mixing is a little too powerful

It's easy to learn "optimal" mixes that oversample from certain pockets heavily. eg, STEM docs are valuable for climbing MMLU & but you don't have infinite STEM docs

We approach mixing as Token Constrained Optimization over diverse evals

🦈Invest in your experimental design!

We create evals better suited for different compute scales, with our "easy" set of tasks+metrics able to support very small scale experiments before switching to our "main" set of evals, on which smaller models are below noise floor

we released Olmo 3! lot of exciting stuff but wanna focus on:

🐟Olmo 3 32B Base, the best fully-open base model to-date, near Qwen 2.5 & Gemma 3 on diverse evals
🐠Olmo 3 32B Think, first fully-open reasoning model approaching Qwen 3 levels
🐡12 training datasets corresp to different staged training

going live with a mukbang tmr 🍱

not happy abt gpt 5.1 update. it's making way more mistakes compared to gpt 5 on basic stuff

latex table formatting errors (straight up missing "&" so columns misaligned, or dropping a whole column, or shifting values by 1 position), feels unusable imo 😒

congrats!! 🎊

omg is this a “simple trick they don’t want u to know”?

picking between 3 checkpoints w/ same benchmark scores but what if one of them is agi

Yay congrats!!

correct framing can make or break research contributions

Research Internship, OLMo Seattle, WA

apply here: job-boards.greenhouse.io/thealleninst...

i answer some FAQs on my site: kyleclo.com/mentorship/

why intern at Ai2?

🐟interns own major parts of our model development, sometimes even leading whole projects
🐡we're committed to open science & actively help our interns publish their work

reach out if u wanna build open language models together 🤝

links 👇

🙌🏻🙌🏻🙌🏻

congrats to our olmo earth team 🌎

small multimodal foundation language models + system for finetuning for important uses like agriculture, wildfire management, conservation & more 🌿

thanks for explaining & sorry it's come to this 😮‍💨

curious your thoughts on other measures like restricting such pieces to senior authors who have a publication record in the surveyed area?

more stories from scholar land 😅

gs tends to have higher cite counts than s2

s2 clusters paper copies & each cluster grants only +1 citation. without proper clusters, each version (eg, preprint vs published) grants citations.

sadly, users can be unhappy when s2 cite counts are lower cuz of this😥

lol yea

also not widely known but a core difference between gscholar & semantic scholar (s2)

gscholar separates UI & data, so when you merge papers, change is only local to your page but not your coauthors

s2 updates the underlying data, and UI reflects ground truth for all users

Google Scholar is manipulatable Citations are widely considered in scientists' evaluation. As such, scientists may be incentivized to inflate their citation counts. While previous literature has examined self-citations and citation ...

lol also arxiv.org/abs/2402.04607

woah guess VLMs for OCR the hottest research topic this week😆 since the first olmOCR, we've been..

🔥training our VLM using RLVR with binary unit test rewards🔥

it's incredibly effective & unit test creation easy to scale w synthetic data pipelines

check it out at olmocr.allen.ai

nice read thx for sharing! I think the piece could use a follow up / complement discussing misaligned incentives that push scientists to compete rather than collaborate (notably, the section on data fragmentation)

bye #colm2025 big fan of the montreal bagels 🥯 hot take I like them better than

lol so much love for prepost-postpre training

any other fans of pre-pretraining?

come say hi at posters this morning for OLMo 2 and fluid benchmarking posters 👋 and dont miss @valentinhofmann.bsky.social's talk in morning #colm2025 @ai2.bsky.social vry proud of my gifs

@josephc.bsky.social @mariaa.bsky.social and I are at poster #21

findings from large scale survey of 800 researchers on how they use LMs in their research #colm2025