Oumi

Oumi - Build Custom Evaluations for any Open or Closed Model in just 50 Lines of Code Use Oumi to build custom evaluations for any open or closed model, in just 50 lines of code

🚀 Custom Evaluations Made Easy with Oumi 🚀

Build custom evaluations for any model in <50 lines of code! 🙌

✅ Simple config change to evaluate models like GPT-4o, Claude 3.7, LLaMA 405B
✅ Learn how we evaluated SOTA LLMs as hallucination classifiers🧠✨

👉 Check it out: oumi.ai/blog/posts/c...

YouTube video by Oumi Evaluate Models with Oumi [walkthrough]

Looking for simple, robust model evaluations? Look no further!

📊Standardize benchmarks
🗣️Generative evals
🔧Customizable evals

Oumi has them all!

Today in our Weekly Walkthrough, @taenin.bsky.social will give you an overview of Oumi’s evaluation framework.

🎥 Watch now: youtu.be/GhHmtjMw-l4

YouTube video by Oumi Let's run batch prediction with Oumi [walkthrough]

Need to label a dataset? Looking to run inference at scale? You’re in luck! 🍀

@taenin.bsky.social tackles Batch Prediction in this edition of our Weekly Walkthrough, showing how you can quickly run inference over your data within the Oumi platform!

🎥 Watch now: youtu.be/3Yg3ycxCEYQ

YouTube video by Oumi Training models with Oumi [walkthrough]

🎥 Watch now: youtu.be/24e6gaCycCI

This week we’re headed to the train station, Platform 135M+! 🚂

In today’s Weekly Walkthrough, @taenin.bsky.social guides us through training with Oumi, showing us how to customize training for your needs (including functionality like PEFT and FSDP). 🙌

📹 Link to watch in thread!

Apparently a lot of you were dreaming about DeepSeek's R1 model 👀

@jgreer-oumi.bsky.social's MiniMath-R1-1.5B has over 750 downloads since we released it last week!

Have you used it to train your own DeepSeek R1 model yet?

On Fridays, we meme.

Credit: @jgreer-oumi.bsky.social

YouTube video by Oumi Walkthrough: Running Inference with Oumi

🎥 Watch now: youtu.be/25ygQkAykpQ

Today we’re kicking off the first of our Weekly Walkthroughs with @taenin.bsky.social 🙌

Each week, Matthew will walk you through how you can use Oumi to make the most of your machine learning workflows ⚙️

In today’s video, Matthew covers model inference in Oumi 🚀

🎥 Link to watch in thread!

oumi-ai/MetaMathQA-R1 · Datasets at Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science.

📈 Data:

huggingface.co/datasets/oum...
huggingface.co/datasets/oum...

💻 Model: huggingface.co/oumi-ai/Mini...

📓 Notebook: github.com/oumi-ai/oumi...

Dreaming about DeepSeek R1 this month? ☁️

We collected 650,000+ R1 prompts and responses for you to train your own DeepSeek-R1 model.

With this data we made MiniMath-R1-1.5B, the top MMLU-Pro-Math model at <=1.5B parameters, achieving an accuracy of 44.4%!

Data, model, and notebook in thread! 🙌

YouTube video by Oumi What is Oumi?

Watch now: youtu.be/11Qk14mSCHk?...

Check out research projects in our discord: discord.gg/oumi

Give us a try on github: github.com/oumi-ai/oumi

What is Oumi? 🤔

We’ve talked a lot about why we built Oumi. Now, let’s dive into what it actually does.

In this video, @taenin.bsky.social walks through the core functionality of Oumi, how it works, and why it matters for AI researchers and developers.

🎥 Link to watch in thread!

Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model Large Language Models (LLMs) with API-calling capabilities enabled building effective Language Agents (LA), while also revolutionizing the conventional task-oriented dialogue (TOD) paradigm. However, ...

Last week, Emre Can Acikgoz (PhD at UIUC) released CoALM 🚀 fully open-source Conversational Agentic Language Models trained using Oumi 🙌

Check out their work!👇

📄 Arxiv: arxiv.org/abs/2502.08820
🌐 Project: emrecanacikgoz.github.io/CoALM/

From the MachineLearning community on Reddit: [D] We built GenAI at Google and Apple, then left to build an open source AI lab, to enable the open community to collaborate and build the next DeepSeek.... Posted by koukoumidis - 5 votes and 0 comments

Do you have questions for the Oumi team on #OpenSource, #DeepSeek, & #FrontierAI?💡

Join us tomorrow, Feb 14 9am-12pm PT for a
Reddit AMA on r/MachineLearning & bring your best questions for our founders on all things open source!

www.reddit.com/r/MachineLea...

Oumi is trending on GitHub 🤯

We’re blown away by the response we’ve received since launching last week.

"The community is ready for 100% truly open AI. Together, we can build the open-source AI the world needs." @koukoumidis.bsky.social

Join us: lnkd.in/gAw3BXGW

#OpenAI #Collaboration #Oumi

Oumi debuts 'open' AI 🌐, DeepSeek under scope 🕵️‍♂️ Oumi strives to be AI's 'Linux moment' with its open-source ambition. Oumi is an open-source AI platform backed by top universities, aiming to support the collaborative development of AI models. Oumi'...

"Global AI collaboration" 👏

Check out Oumi in today's Daily Rede tech & AI newsletter: Oumi debuts 'open' AI 🌐, DeepSeek under scope 🕵️‍♂️ Read more: rede.io/newsletters/...

🚀

Introducing Oumi 🚀 a community of researchers and developers united in their mission to make frontier AI more open, collaborative, and accessible. Join us in building the platform, models, and tools: let’s shape the future of AI. #oumi #opensource #collaboration

github.com/oumi-ai/oumi