Joe Hamman

Super excited to see #icechunk v1.0 ship today. Stable format, stable API, and ready for production. Take it for a spin and let us know how it goes. 🚀🚀🚀

Just back from #CNG2025, I wrote up some thoughts on the traction the @zarr.dev format is gaining in the broader geospatial industry (specifically Earth Observation). I also summarize what features and tools the user and developer communities are asking for to fully leverage Zarr. 👇👇👇

I'll be at the CNG conference in Snowbird next week. I wrote a short blog post about what the Earthmover team will be up to.

tldr; we'll be talking about @zarr.dev, #icechunk, @xarray.bsky.social and cloud-native data cubes.

Details in the blog post 👇

Most people think of @zarr.dev as a "file format". With #Icechunk, we've turned Zarr into a database. @functionth.bsky.social's post shows how Icechunk can be used to solve a problem where transactional databases are often required.

Learning about Icechunk consistency with a clichéd but instructive example - Earthmover In this post we’ll show what can happen when more than one process write to the same Icechunk repository concurrently, and how Icechunk uses transactions and conflict resolution to guarantee consisten...

1/ 🚨 New Blog Post Alert: "𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝐴𝑏𝑜𝑢𝑡 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝐶𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑎 𝐶𝑙𝑖𝑐ℎ𝑒́𝑑 𝑏𝑢𝑡 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑣𝑒 𝐸𝑥𝑎𝑚𝑝𝑙𝑒" 🏦🔁

👉 Read it here: earthmover.io/blog/learnin...

There's still time to sign up for tomorrow's webinar on ⚡Flux ⚡

👇
share.hsforms.com/1FLrE9CpwSLq...

@zarr.dev and #icechunk are amazing but they are not magic. They are part of a thoughtfully designed cloud-native data architecture. @tegnicholas.bsky.social peels back the covers on cloud-optimized scientific data formats in our latest "Fundamentals" post 👇

We found similar results when we first benchmarked #icechunk. Our conclusion: doing IO with a Rust backend is much faster than Python.

👇Really exciting to see @kylebarron.dev's Obstore backend for Zarr-Python ship today.

Announcing Flux: The API Layer for Geospatial Data Delivery - Earthmover TLDR Earthmover’s new product–Flux–adds a whole new layer to our platform. Flux allows you to serve geospatial data from Arraylake via standard API protocols–including WMS (web map service), EDR (envi...

📝 More details: earthmover.io/blog/announc...
📅 Webinar April 22: share.hsforms.com/1FLrE9CpwSLq...
💬 DMs open — happy to chat or schedule a demo.

No more bespoke APIs. No glue code.
Just fast, interoperable access via tools you already use — QGIS, ArcGIS, web maps, Python, R, MATLAB, and more.
Flux is a cheat code for building data products.

Flux serves data directly from Icechunk via standard APIs like EDR, WMS, and OPeNDAP.
It’s stateless, scalable, and cloud-native.
From raw data to production-grade API endpoints in seconds.

We built Flux to solve a core pain point: turning massive geospatial datasets into usable APIs was slow, brittle, and distracting.
Flux makes it trivial for any data producer to become a data provider.

🚀 Big launch today: introducing Flux — a game-changing way to serve geospatial data via standard APIs in seconds. Built for scale, speed, and simplicity.

Training AI models at scale from data stored in cloud object storage requires thinking carefully about both bandwidth and concurrency. In this post, @functionth.bsky.social get’s into the details of concurrent reads at scale, showing how #Icechunk and S3 can easily scale beyond 200k requests/second!

Exploring Icechunk scalability: untangling S3's prefix story | Earthmover We show Icechunk can scale to extremely high concurrency levels, and explain how it achieves this in modern object stores.

📣 Blog post alert! 𝐄𝐱𝐩𝐥𝐨𝐫𝐢𝐧𝐠 𝐈𝐜𝐞𝐜𝐡𝐮𝐧𝐤 𝐬𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲: 𝐮𝐧𝐭𝐚𝐧𝐠𝐥𝐢𝐧𝐠 𝐒𝟑'𝐬 𝐩𝐫𝐞𝐟𝐢𝐱 𝐬𝐭𝐨𝐫𝐲. This technical post by @functionth.bsky.social dives deep into the internals of how S3 shards data, showing that distributed Icechunk can easily perform 230,000 object reads/sec and beyond. earthmover.io/blog/explori...

We’ll look into it!

We often see folks try to convince tabular data tools to perform well with multi-dimensional array data. This post by @rabernat.bsky.social explains, from first principles, why this rarely works. Its a good one! 👇👇👇

Super exciting to see @copernicusecmwf.bsky.social starting to adopt @zarr.dev!

🚨 New blog post 🚨

In it, we show off our recent work deploying #icechunk on top of #NASA's existing archives of Earth observation data. The results: 100x speed up when extracting time series from existing datasets stored as netCDF.

This session is going to be a blast! If you are headed to CNG next month (and you should be!), consider joining us for this workshop on @xarray.bsky.social , @zarr.dev , and #icechunk. 👇👇👇

1/ As a data platform company, we get a unique view of how other companies use data. In light of the recent cuts at @noaa.gov, our CEO, @rabernat.bsky.social, reached out to our customers to get their perspective on the value of NOAA’s data and services.

I'll be giving a short update tomorrow on the state of Zarr-Python following the 3.0 release back in January. Swing by the Pangeo showcase to catch the latest progress.

@zarr.dev

Denver Geospatial Happy Hour · Luma Come have a drink, grab a bite, and nerd out with the Earthmover team on geospatial data. Earthmover will provide food and drink tickets. Our event will be…

Denver folks — join us for a Geospatial Happy Hour on Wednesday, March 12th at 7 PM MST. Come enjoy drinks, tacos, discussing geospatial data, and meeting the Earthmover team. Register: lu.ma/o6r39p6j

We now have a public slack channel for VirtualiZarr development and questions, feel free to join!

earthmover-community.slack.com/archives/C08...

Very, very stoked to have @tegnicholas.bsky.social join the Earthmover team!

I'm curious what sort of issues you are seeing. I'm able to open datasets on s3 without any problems.

Pre race team photo

I rode the race with my good friend Nico! Together, we rode about 320 miles over 24 hours! www.strava.com/activities/1...

24HOP Open Men Duo Podium

Here on Bluesky, I follow two topics: 1) software & data infrastructure and 2) cycling. I mostly post about software but today, I can share a bit about a fun cycling adventure of my own 🚵‍♂️.

Last week, I got to ride in the 24 Hours in the Old Pueblo - placing 4th in the Open Men Duo category:

Zarr-Python 3 included a bunch of performance improvements that are now landing in @xarray.bsky.social. Check out @davisvbennett.bsky.social's post on what we changed in Xarray/Zarr and how it is speeding up the "time to first byte" for large datasets 👇