Matthew Feickert's Avatar

Matthew Feickert

@matthewfeickert.com.bsky.social

Research scientist at UW-Madison data science institute working on #LHC #physics and data science with the ATLAS experiment @ CERN and IRIS-HEP. PhD @ SMU

716 Followers  |  488 Following  |  448 Posts  |  Joined: 07.07.2023  |  2.3502

Latest posts by matthewfeickert.com on Bluesky

Preview
ENH: Add Linux aarch64, ppc64le, and macOS arm64 builds by matthewfeickert Β· Pull Request #1 Β· conda-forge/r-gdalraster-feedstock Enable builds for platforms: linux-aarch64, linux-ppc64le, osx-arm64. Bump build number. MNT: Re-rendered with conda-build 25.5.0, conda-smithy 3.51.1, and conda-forge-pinning 2025.07.29.16.05.27 ...

github.com/conda-forge/...

30.07.2025 07:51 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

So

```
pixi add r-gdalraster
```

is all you need to get going.

30.07.2025 07:28 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Preview
GitHub - conda-forge/r-gdalraster-feedstock: A conda-smithy repository for r-gdalraster. A conda-smithy repository for r-gdalraster. Contribute to conda-forge/r-gdalraster-feedstock development by creating an account on GitHub.

github.com/conda-forge/... exists now. It should soon have multi-platform builds for Linux and macOS added, and looks like from info from Daniel Nachun github.com/conda-forge/... there might be some additional work on getting this ecosystem of R tools to work on Windows.

30.07.2025 07:28 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

Linux containers != packaging. bsky.app/profile/matt...

Also, correct, you should have zero patience for that dev cycle, though until maybe 3 years ago this was realistically a way to do it for complex dependencies (e.g. CUDA). Now CUDA environments can be trivially handled with Pixi + conda-forge.

23.07.2025 13:34 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Strongly disagree with this. I've become far more of an expert in Linux containers in the last 7 years then I ever expected, and trying to compose environments between multiple images is not a viable solution for most people. Also, Linux containers != packaging! They're _distribution_ models.

23.07.2025 13:27 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 2

You are correct that the feedstock creation for R on @conda-forge.org _needs_ to improve quite a bit (github.com/bgruening/co...). And once again with feeling, conda-forge is NOT Anaconda.com. There's no need to be using the defaults channel and can be trivially avoided.

23.07.2025 13:25 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

System level binaries is an immediate non-starter. That is not how you build multiple computational environments.

23.07.2025 13:21 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Seems from this subthread that there's a potential for fixes in the future wrt PPM: bsky.app/profile/lach...

23.07.2025 05:56 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

And for all the ones that don't? Which is most.

23.07.2025 05:30 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Without some big banner noting that there's additional required steps, those docs are wrong.

23.07.2025 05:29 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0
Posit Package Manager

If that's the case, that's good to hear. But I'd gently suggest that Posit fix the docs to note that Linux binary support is _not_ as described in the docs that you hit when you land on the PPM setup page.

packagemanager.posit.co/client/#/rep...

23.07.2025 05:29 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 1

And so users should be able to get binaries for everything that Posit has built in PPM without having to run additional configuration scripts in advance, correct?

23.07.2025 05:19 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0
Post image Post image

Thanks for the detailed summary, as that combination of things works. So if I understand things correctly, this is the required way to do things _now_, but whenever the next release of Package Manager happens there will be new docs on how to just provide an index URL in ~/.Rprofile to get binaries?

23.07.2025 05:19 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Again, real questions: Do R users primarily only use laptops then to do their work? Are people not running on clusters ever?

23.07.2025 04:34 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
https://r-universe.dev/ Welcome to R-universe!

Aren't r-universe.dev indexes mutable? Doesn't that defeat any notion of reproducible environments? Or in practice is every build never removed?

23.07.2025 04:32 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Add a new CRAN-like repository β€” repo_add Add a new repository to the list of repositories that pak uses to look for packages.

The docs pak.r-lib.org/reference/re... make no mention of what the default behavior is. How should users know they need to use register a new repo? Is this your default workflow to always need to ask for "PPM@latest"?

23.07.2025 04:27 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

* Correction. Ubuntu 24.04, though hopefully that doesn't matter at all.

23.07.2025 03:14 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

Thanks very much for engaging and trying to point to examples of tools that you enjoy and help you. However, pak (or more posit package manager) fails to be able to provide binaries and so is not a solution: bsky.app/profile/matt...

23.07.2025 03:09 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

Other important piece that I failed to mention: conda-forge was created in 2015 and started getting what I would call good around 2000, and then got _very_ good around 2022.

23.07.2025 03:06 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

I'll comment here as that could be taken the wrong way: People obviously do real science on macOS and Windows! But scientific computing clusters are Linux, so R users doing computing at research institutions are either going to have to or _want_ to use Linux machines at some point.

23.07.2025 03:03 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 0    πŸ“Œ 0

I would be delighted for someone to show me that I've been missing a super obvious solution that everyone else knows, but from earlier responses to my original post months ago that doesn't seem to be the case. Thank you for taking so much time to engage and to point to things you use!

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

I am truly very happy to hear that for so many people on macOS and Windows the binary support for R packages on CRAN is excellent. That's great and should be celebrated! The question to me is "What do people doing serious computing with R on Linux (e.g. clusters) do for packages?"

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

RE: your past experience with conda: 100%. As a C++ & Python person I remember playing with conda in 2014 and rejecting it as the wheel support of Python packages started getting very good around the same time. I used to describe myself as a conda hater until about 2 years ago when I found Pixi!

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

My question then loops back to the original post I had. Why are people content with having basically no binary support for Linux?

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

> users generally don’t use the terminal much, so the idea of a separate package manager application seemed confusing.

That's super valid! I fully appreciate that if installing packages _inside_ of your code is the norm, then _external_ installation is very strange.

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

> What is the workflow you would like to see in R?

I'm not a real R user, so my views here aren't really important, but in general I'd like for people to be able to declaratively state what they need and have that and all dependencies get installed as securely built binaries very fast.

23.07.2025 03:02 β€” πŸ‘ 2    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

> I spend 5-10 minutes with pak to install dependencies, that feels fine.

I commend you for your patience. I am expecting full > 1GB environments of potentially complex dependencies to be solved, locked, downloaded, and installed in well under 1 minute.

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0
Post image Post image

* What does

```
pak::pkg_install("dplyr")
```

do? It builds 15 packages from source. This is for a Ubuntu 22.04 x86 Linux container with R v4.5.x. So pak is not getting built distributions, because they don't exist. This is not a solution.

23.07.2025 03:02 β€” πŸ‘ 0    πŸ” 0    πŸ’¬ 3    πŸ“Œ 1
R Pak | Anaconda.org

* pak is on conda-forge (anaconda.org/conda-forge/...) but I'm going to try to use it how you are suggesting, which is off to a bad start because there's no binaries for it for Linux and required building from source 17 other packages. This is fine for me. Imagine you're a first year Ph.D. student.

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 1    πŸ“Œ 0

To answer your questions across multiple posts:

* conda-forge gives you multi-platform built distributions of packages that can optionally have microarch optimizations. Having immutable built distributions is super important for multiple reasons, including scientific reproducibility.

23.07.2025 03:02 β€” πŸ‘ 1    πŸ” 0    πŸ’¬ 2    πŸ“Œ 0

@matthewfeickert.com is following 20 prominent accounts