@pola.rs.bsky.social
Dataframes powered by a multithreaded, vectorized query engine, written in Rust.
The third Polars Meetup is confirmed! On 17 September, we will organize the next in-person meetup at @quantco.com in Munich, Germany.
Topics:
- Parquet reader improvements
- Migrating pipelines using Dataframely, Quantco's open-sourced schema validation tool.
RSVP: www.meetup.com/polars-meetu...
The videos of our recent San Francisco meetup are in!
- Ritchie Vink covered the new streaming engine and shared updates on Polars Cloud and the upcoming distributed engine.
- Vyas Ramasubramani shared how GPU accelerated Polars works.
youtube.com/watch?v=fYi9...
youtube.com/watch?v=fYi9S6
"Came for the speed, stayed for the syntax." Once users have learned our API, they love it. And it shows.
Polars is the 3rd most admired rising tech in this years StackOverflow developer survey.
survey.stackoverflow.co/2025/technol...
The pre-release of polars 1.32 is out. This is a big release with a lot of internal improvements.
Please consider trying it so that we can patch any regression before we ship the release next week.
github.com/pola-rs/pola...
Demo of Formulaic with native Polars support
✨ New Formulaic release (1.2), featuring native support for @pola.rs !
🌊🦄 Powered by Narwhals
We're hosting our second meetup in San Francisco on July 24th.
Ritchie Vink will introduce Polars Cloud, the platform to scale Polars remotely. Vyas Ramasubramani from NVIDIA will be talking about the internals of accelerating your Polars queries with the GPU engine.
RSVP: lu.ma/60b6wfs8
🐍📰 How to Group Data Using Polars .group_by()
Start using Polars .group_by() to make sense of your data. This tutorial shows you how to group, aggregate, and reveal hidden insights with hands-on examples
#python
A stylized polar bear looks down at a laptop displaying colorful charts and graphs. To the left, text reads "Data Validation Libraries for Polars (2025 Edition)". The background is a solid blue with a subtle grid pattern.
@richmeister.bsky.social has released the "Data Validation Libraries for Polars (2025 Edition)" guide!
Reviews 5 #Python validation libraries that work with @pola.rs DataFrames: Pandera, Patito, Pointblank, Validoopsie, & Dataframely.
Read it on the Pointblog: posit-dev.github.io/pointblank/b...
The recent introduction of Unified Virtual Memory (UVM) for GPUs allows users to process data larger than the GPU's VRAM.
Learn how UVM works in more detail and how to optimize the configuration for your use cases, including performance implications.
pola.rs/posts/uvm-la...
In that blog they explain they did manual join order optimization for Dask only.
We don't allow that as that can differ hugely in rows produced by joins and would be an unfair comparison.
Polars 1.31 is released. With as highlight DataType expressions✨. These allow you to get the datatype of an expression dynamically.
See the full changelog here.
github.com/pola-rs/pola...
Want something more with DataType expressions, here is the RFC:
github.com/pola-rs/polars
We enabled pyarrow datatypes. There is no GIL contention on them. If you think we should adapt the settings, please let us know, we want to run with optimal settings.
18.06.2025 14:32 — 👍 3 🔁 0 💬 0 📌 0
We've updated our benchmarks run. It has been more than a year since we ran them. Since then we've designed and implemented a complete novel streaming engine that can deal with Polars' data model.
The future of Polars looks bright and very, very fast!
pola.rs/posts/benchm...
Course title slide with dark blue background showing 'FREE COURSE' in green text, 'Introduction to Polars' as the main white heading, and logos for DataCamp and Polars at the bottom.
We've partnered with @datacamp.bsky.social to create an interactive Polars course.
Learn the fundamentals and get familiar with our API through hands-on exercises. The course is available for everyone and free until the end of August.
Start the free course here: www.datacamp.com/courses/intr...
🐍📰 How to Group Data Using Polars .group_by() realpython.com/polars-group...
19.05.2025 14:01 — 👍 4 🔁 1 💬 1 📌 0
Polars has gotten 4x faster than Polars! 🚀
In the last months, the team has worked incredibly hard on the new-streaming engine and the results pay off. It is incredibly fast, and beats the Polars in-memory engine by a factor of 4 on a 96vCPU machine.
Diagram showing examples of applying horizontal operations to a dataframe with random numerical values. The example shows the expressions max_horizontal, sum_horizontal, mean_horizontal, and cum_sum_horizontal. The first three produce a numerical column and the expression cum_sum_horizontal produces a struct column with as many fields as there are input columns/series.
Polars provides a number of xxx_horizontal operations.
These expressions perform computations across columns. (Or along rows, depending on how you look at it.)
If your horizontal operation isn’t implemented, you can use the general-purpose fold.
Metro.digital used Polars to simplify the KPI creation process across 19 countries, cutting down the processing time while reducing compute resource requirements.
Dr. Patrick Bormann shares his experience and some of the optimization techniques he and his team used:
pola.rs/posts/case-m...
Imagine a 98% cost reduction by switching to Polars!
@jeroenjanssens.com and Thijs Nieuwdorp share how Xomnia did it for Alliander, thanks to a lazy and streamlined API and smart caching.
Watch the @pola.rs meetup recording: www.youtube.com/watch?v=7DV6...
#PythonProgramming #PythonDev
During our first Meetup, Jeroen and Thijs shared how they migrated a large codebase to Polars at a utility company and achieved a 98% cost reduction processing more data on smaller machines.
Learn from their best practices to migrate your codebase to Polars: www.youtube.com/watch?v=7DV6...
The recordings of our first Meetup are now available. Watch Orson Peters' talk to learn about the internals of the new streaming engine and its performance benefits.
Watch here: www.youtube.com/watch?v=Ndil...
This Thursday is our first (hybrid) meetup with two talks:
- Introduction of the new streaming engine
- Lessons learned from migrating a large code base to Polars
First talks start at:
5:45 PM CEST | 11:45 AM EDT | 8:45 AM PDT
RSVP to join: www.meetup.com/polars-meetu...
Combining strings in @pola.rs is easy!
31.03.2025 04:01 — 👍 2 🔁 1 💬 1 📌 0
Computers are good at capturing timestamps, but they do it in Unix time, which is hard for humans to understand.
@pola.rs allows you to convert Unix time to human-readable datetime.
#100DaysOfPolars
#datasky
When displaying budget figures, you may want to include a currency symbol.
@pola.rs allows you to easily add any currency symbol to your figures.
#100DaysOfPolars
#datasky
Tabular #data is easier to process when columns contain singular values. But sometimes, data is stored as a list in one column.
With @pola.rs, you can make every item in the list as it's own row.
#100DaysOfPolars
#datasky
Sometimes, you want the columns in your table to be in a specific order. But typing out the names can be tedious
Luckily @pola.rs has a way that doesn't require typing names.
#100DaysOfPolars
#datasky
Image showing polars code for data analysis.
Always remove whitespace before doing a unique count of values to get an accurate count.
In @pola.rs, there's a function to do just that!
#100DaysOfPolars
#datasky
Diagram showing how the functions date_range, datetime_range, and time_range, can be used to generate series of consecutive temporal values. The interval between consecutive values can be changed with the parameter interval and the two endpoints may or may not be included in the generated range, depending on the value the parameter closed is set to.
Polars provides 3 functions you can use to generate temporal ranges:
date_range, datetime_range, and time_range.
These can be executed eagerly or lazily.
You can also customize the interval between consecutive values and whether the start/end points are included.
Pivot Dataframes with @pola.rs
17.03.2025 21:47 — 👍 3 🔁 1 💬 1 📌 0
