Stephan Hoyer

Do you take it yourself?

I think the problem is the algorithm. BlueSky's lack of a recommendation engine means that if you're not posting all the time, your stuff doesn't get seen.

Why PyTorch is an amazing place to work... and Why I'm Joining Thinking Machines In which I convince to you to join either PyTorch or Thinking Machines!

The "ungamable impact" of OSS really resonates with me:
www.thonking.ai/i/158277004/...

Sadly it does not necessarily align with what makes for a sucessful career in Big Tech. But it's worth trying anyways! :)

I think it's just about readability with small font, the same reason why printed newspapers use many columns.

The losses here should be marked as millions not billions, right?

Pretty much anything that you can write in high level array code like NumPy is very fast in JAX. Only intrinsically very loopy code is (relatively) slow, but JAX has excellent support for writing custom kernels in lower level languages.

AD compatible Python is at the cutting edge of performance these days with it's central role in large-scale AI training.

In my experience (mostly geophysical fluid dynamics) JAX has comparable perf to modern Fortran on CPUs, with a much easier path to GPUs and multi-device code.

Those are tiny chunks! Does that reduce max throughput for analytics use-cases compared to larger chunks?

Such exciting news!

For anyone who has tried the new sharding feature -- do you have any guidance on optimal shard sizes, if I want more flexibility in access patterns but still optimal throughput?

Is there a link between #ClimateChange & increasing risk/severity of #wildfire in California--including the still-unfolding disaster? Yes. Is climate change the only factor at play? No, of course not. So what's really going on? [Thread] #CAfire #CAwx #LAfires iopscience.iop.org/a...

This is a huge milestone for cloud-native big scientific data!

The Risky Business of Predicting Where Climate Disaster Will Hit Climate tech companies can calculate the chances that a flood or wildfire will ravage your home. But what if their odds are all different?

Hi, thanks for the mention. Here's a 7-day paywall-free link to the main feature: www.bloomberg.com/graphics/202...

Correcting Weather and Climate Models by Machine Learning Nudged Historical Simulations Nudging an atmospheric model toward observations is a good way to estimate state-dependent biases Machine learning of state-dependent biases improves hindcast skill of a coarse-resolution general...

This paper by Watt-Meyer et al is a good example of "Error-based learning:" agupubs.onlinelibrary.wiley.com/doi/10.1029/...

ECMWF has also done similar work on top of IFS's data assimilation system.

¡AI Caramba! Rapid progress in the use of machine learning for weather and climate models is evident almost everywhere, but can we distinguish between real advances and vaporware? First off, let's define some...

Some thoughts on the use of AI/ML in climate modeling...

@realclimate.org

¡AI Caramba! www.realclimate.org/index.php/ar...

We have a few pre-computed climatologies in WeatherBench2: weatherbench2.readthedocs.io/en/latest/da...

We have a few other updates to share as well, which can be found in the inaugral edition of the NeuralGCM newsletter:
groups.google.com/g/neuralgcm-...

The biggest one is that NeuralGCM models are now freely available for everyone to use, including for commercial purposes!

Neural general circulation models optimized to predict satellite-based precipitation observations Climate models struggle to accurately simulate precipitation, particularly extremes and the diurnal cycle. Here, we present a hybrid model that is trained directly on satellite-based precipitation obs...

Can incorporating AI improve precipitation in global weather and climate models?

Yes! In the latest NeuralGCM paper, we show that training on satellite-based precipitation results in significant improvements over traditional atmospheric models:
arxiv.org/abs/2412.11973

Please reach out if you want to chat about anything related to AI modeling, NeuralGCM, JAX or Xarray. Also see Eni's poster on xarray.DataTree on Thurs: agu.confex.com/agu/agu24/me...

Interested in AI weather/climate modeling at #AGU24?

I'll be giving an overview talk on NeuralGCM at 11:30am Wed at the Google booth, and an talk on modeling precipitation with NeuralGCM at 4:25pm Wed in the session A34A.

When I hear "ML" I tend to think of old school (i.e., scikit-learn) machine learning, which is great but much less powerful than deep learning. So I would opt for "AI weather models" though that misses quite a bit of nuance.

This diagram is accurate historically, but recently AI seems to have become synonymous with deep learning.

The bottleneck for traditional models is data movement within the CPU, not data transfer to disk -- physics based simulations do too little compute per byte (low arithmetic intensity) to fully utilize modern hardware.

AI is way better in this respect. It's easy to use lots of FLOPs on big matmuls!

Unlimited potential, zero bugs!

There's nothing like the feeling of starting a codebase from scratch.

To my knowledge, there are no limits on accessing ARCO-ERA5 or other public datasets stored in Google Cloud Storage. You don't even have to be logged in!

One of my favourite data discoveries this year: Google's mind-blowing ARCO-ERA5 dataset: hourly data for ~300 climate variables, available globally from 1940! 🤯

Loadable with a single line of Python code from a single cloud-friendly Zarr file! Below: a month of wind waves + swell: 🌊

Fast JAX simulations of all the PDEs--whee!

Let me know if you're headed to AGU :)

Any recommendations for those of us building neural PDE models?