's Avatar

@marcogorelli.bsky.social

Senior Software Engineer at Quansight Labs Pretty focused on Narwhals and Polars these days, generally interested in dataframes and time series

198 Followers  |  189 Following  |  63 Posts  |  Joined: 28.10.2024  |  2.2169

Latest posts by marcogorelli.bsky.social on Bluesky

Example of `.name` chaining with Polars

Example of `.name` chaining with Polars

Example of `.name` chaining with Narwhals

Example of `.name` chaining with Narwhals

✨ New @pola.rs feature - you can now chain `name` expressions!

πŸŒŠπŸ¦„ As of Narwhals 2.0 (yes, we've released 2.0 πŸš€), this feature is available there as well! Little example below with our friend @duckdb.org

12.08.2025 17:28 β€” πŸ‘ 3    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
temporary solution added to pandas 12 years ago

temporary solution added to pandas 12 years ago

πŸ’‘ "There's nothing for permanent than a temporary fix"

Case in point, this pandas comment which was added [checks git log] ... 12 years ago πŸ™ˆ

07.08.2025 20:16 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

"If the U.S. can't get its greenhouse emissions under control, they need to face real consequences" said the rest of the world.

07.08.2025 19:57 β€” πŸ‘ 494    πŸ” 157    πŸ’¬ 23    πŸ“Œ 4
Screenshot of 17-year-old commit message in NumPy

Screenshot of 17-year-old commit message in NumPy

πŸ’‘ Today in open source: a 17-year-old NumPy commit promising that a new feature will be coming soon

Any day now..

29.07.2025 16:35 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
gt-extras demo

gt-extras demo

😍 @posit.co have done it again

πŸ§™ gt-extras offers even more functionality for their table renditions of @pola.rs and pandas dataframes!

πŸŒŠπŸ¦„ Using Narwhals for some dataframe logic

26.07.2025 15:01 β€” πŸ‘ 10    πŸ” 2    πŸ’¬ 0    πŸ“Œ 0

solarized light

20.07.2025 13:26 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Meme of panicked Astronomer CEO caught on cam with the head of HR, with "When you find a new data science library" and "When you realise it only supports pandas" text overlaid

Meme of panicked Astronomer CEO caught on cam with the head of HR, with "When you find a new data science library" and "When you realise it only supports pandas" text overlaid

πŸ’‘ Solution: use Narwhals for universal dataframe support πŸŒŠπŸ¦„

18.07.2025 10:56 β€” πŸ‘ 4    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Demo of Formulaic with native Polars support

Demo of Formulaic with native Polars support

✨ New Formulaic release (1.2), featuring native support for @pola.rs !

πŸŒŠπŸ¦„ Powered by Narwhals

15.07.2025 18:40 β€” πŸ‘ 10    πŸ” 2    πŸ’¬ 1    πŸ“Œ 1
Marco Gorelli - How Narwhals brings Polars, DuckDB, PyArrow, & pandas together | PyData London 25
YouTube video by PyData Marco Gorelli - How Narwhals brings Polars, DuckDB, PyArrow, & pandas together | PyData London 25

🎒 When the Q&A goes off-script 🀣

πŸ‡―πŸ‡΅ Japanese math rock, πŸ…ΏοΈ porpoises and unfortunate aliases, and 🎀 Taylor Swift:

πŸ₯ˆAll of these unexpectedly came up in this fun session, which is currently the 2nd most viewed of the conference

πŸ™ Thanks @pydatalondon.bsky.social for inviting me!

09.07.2025 14:41 β€” πŸ‘ 8    πŸ” 2    πŸ’¬ 0    πŸ“Œ 0
Preview
Adopt Narwhals to support multiple eager dataframe Β· Issue #16518 Β· mlflow/mlflow Description Follow up from #10292 (comment) What's Narwhals? Narwhals is a lightweight compatibility layer between dataframe libraries, with negligible overhead. Its API is a subset of the polars o...

github.com/mlflow/mlflo...

01.07.2025 15:49 β€” πŸ‘ 3    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
MLFlow maintainer announcing that they merged the Polars dataset PR

MLFlow maintainer announcing that they merged the Polars dataset PR

πŸ₯³ The next version of @mlflow.org will support @pola.rs !

😩 Slightly disappointingly, they do so by converting to pandas

πŸŒŠπŸ¦„ Could Narwhals help make MLFlow gain native support for Polars? Link to the discussion below πŸ‘‡

01.07.2025 15:49 β€” πŸ‘ 3    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

sure - how do you know

21.06.2025 13:40 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0
PR which added Ibis support to Narwhals

PR which added Ibis support to Narwhals

Demo of Ibis via Narwhals

Demo of Ibis via Narwhals

πŸ™€ It's scary to merge a 2,000+ line PR!

πŸ˜‡ But, this one was so thorough that we just had to ship it

πŸ† Thanks to an incredible effort by Ryan Whitten, Narwhals supports Ibis!

πŸ’‘You can now use Narwhals with all of Ibis' backends, plus some extra, whilst keeping @pola.rs expressive API!

19.06.2025 18:59 β€” πŸ‘ 3    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
Release Python Polars 1.30.0 Β· pola-rs/polars πŸš€ Performance improvements Switch eligible casts to non-strict in optimizer (#22850) Allow predicate passing set_sorted (#22797) Increase default cross-file parallelism limit for new-streaming mul...

github.com/pola-rs/pola...

06.06.2025 18:08 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Demo of `list.filter` in Polars

Demo of `list.filter` in Polars

✨ @pola.rs now supports `list.filter`

πŸͺ„ The `list` namespace is now even more turbocharged ⚑

πŸ‘‡ Full release notes for 1.30

06.06.2025 18:07 β€” πŸ‘ 6    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

sometimes that's all you need πŸ˜€

26.05.2025 19:17 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Example of using PySpark via Narwhals

Example of using PySpark via Narwhals

✨ PySpark 4.0 is out

πŸŒŠπŸ¦„ And it's already supported by Narwhals, including PySpark connect (thanks Francesco Bruzzesi!)

26.05.2025 19:09 β€” πŸ‘ 1    πŸ” 1    πŸ’¬ 0    πŸ“Œ 0

Looks like a good decision, given she committed fraud

26.05.2025 09:57 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
GitHub - narwhals-dev/narwhals: Lightweight and extensible compatibility layer between dataframe libraries! Lightweight and extensible compatibility layer between dataframe libraries! - narwhals-dev/narwhals

πŸ‘‰πŸŒŸ Narwhals: github.com/narwhals-dev...

πŸ“š Narwhals and SQL: narwhals-dev.github.io/narwhals/gen...

22.05.2025 14:56 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Demo of forward-fill with Narwhals

Demo of forward-fill with Narwhals

Forward-fill as DuckDB SQL

Forward-fill as DuckDB SQL

✨ Narwhals now supports `fill_null` with forwards and backwards strategies

🎬 Here's a demo of a fill-forward function which supports PySpark, @pola.rs , @duckdb.org , pandas, cuDF, and more!

🎁 Bonus: curious about what this looks like as DuckDB SQL? We got you covered 😎

22.05.2025 14:55 β€” πŸ‘ 2    πŸ” 1    πŸ’¬ 1    πŸ“Œ 0
Demo of Narwhals dataframe-agnostic function which supports PySpark

Demo of Narwhals dataframe-agnostic function which supports PySpark

Plot of PySpark dataframe after converting it to PyArrow

Plot of PySpark dataframe after converting it to PyArrow

SQL generation from Polars syntax

SQL generation from Polars syntax

✨ Narwhals now supports PySpark

πŸŽ‡ If you have a dataframe-agnostic function, you can pass in `pyspark.sql.DataFrame`

πŸ“ˆ Here's a demo, made with @marimo.io

🎁 BONUS feature: combine with SQLFrame, to generate SQL from @pola.rs syntax πŸͺ„

09.05.2025 13:41 β€” πŸ‘ 1    πŸ” 1    πŸ’¬ 0    πŸ“Œ 0
Narwhals tests in cuDF's CI

Narwhals tests in cuDF's CI

πŸ› οΈ NVIDIA's cuDF library now runs the Narwhals test suite as part of its CI process

πŸ™€

07.05.2025 12:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

in netherlands?

02.05.2025 15:49 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

I still name my Marimo notebooks `untitled42.py` πŸ™ˆ

30.04.2025 11:57 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Example of Polars is_business_day

Example of Polars is_business_day

✨ New Polars feature: `is_business_day`

🌴 Use this to determine which dates fall on holidays, blazingly fast

πŸ‡©πŸ‡ͺ Using Germany as an example because I was just there to give a talk at @pyconde.bsky.social - thanks for having me and for putting on such a fun event!

25.04.2025 17:12 β€” πŸ‘ 1    πŸ” 1    πŸ’¬ 0    πŸ“Œ 0
Comparison of before vs after for Polars autocompletion - since 1.27.1, it gives correct suggestions!

Comparison of before vs after for Polars autocompletion - since 1.27.1, it gives correct suggestions!

🎈✨ Feeling pumped for this new @pola.rs feature!

It's not a perf improvement, and it's not a new expression

It's...working auto-complete in Jupyter / IPython / Marimo! πŸ™€πŸ˜±

πŸ™ Credit to @krassowski.bsky.social for debugging and Alex Beedie for submitting a fix!

πŸ§‘β€πŸ« This is gonna be great for teaching

11.04.2025 15:24 β€” πŸ‘ 4    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

which one?

28.03.2025 19:02 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
List normalisation in DuckDB

List normalisation in DuckDB

List normalisation in Polars

List normalisation in Polars

πŸ“š List normalisation: Polars vs DuckDB syntax

πŸ»β€β„οΈ Polars: divide the column by its list sum. Expressive as always 🎨

πŸ¦† DuckDB: `a / list_sum(a)` would error...but you can write a list comprehension πŸ˜±πŸ™€

19.03.2025 16:40 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Darts release notes

Darts release notes

Bokeh release notes

Bokeh release notes

✨✨ 2 big Narwhalified releases this week!

🎯 For time series enthusiasts, Darts now supports @pola.rs / PyArrow / cuDF / Modin via Narwhals

🎨 For visualisation enthusiasts, same story for @bokeh.org

πŸ¦„πŸŒŠ Underwater unicorn magic

14.03.2025 16:16 β€” πŸ‘ 4    πŸ” 1    πŸ’¬ 0    πŸ“Œ 0
Preview
Bokeh 3.7 is released! Check out the details in the blog post! For full details about this release, you can check out the release notes here: Releases β€” Bokeh 3.7.0 Documentation

We are excited to be here on @bsky.app and excited to announce the release of Bokeh version 3.7! Some great new features like windowed auto-ranging, support for more DataFame types, and new UI componentsβ€”read all about it on our latest blog post

discourse.bokeh.org/t/bokeh-3-7-...

12.03.2025 21:24 β€” πŸ‘ 5    πŸ” 5    πŸ’¬ 0    πŸ“Œ 0

@marcogorelli is following 20 prominent accounts