Collin Berke

Note to self: It's been awhile since I've needed to do back indexing to return a single row of data while using #RStats.

So, here's a Base R and dplyr refresher 👇

#dataBS

Comprehension Debt: The Ticking Time Bomb of LLM-Generated Code An effect that’s being more and more widely reported is the increase in time it’s taking developers to modify or fix code that was generated by Large Language Models. If you’ve wo…

Interesting framing of the use of LLMs for coding in a post I recently bumped into: codemanship.wordpress.com/2025/09/30/c... (via news.ycombinator.com/item?id=4542...).

I'm certainly not AI-averse when performing coding tasks. But, 'Comprehension Debt' is something to be mindful of.

data.tree: General Purpose Hierarchical Data Structure Create tree structures from hierarchical data, and traverse the tree in various orders. Aggregate, cumulate, print, plot, convert to and from data.frame and more. Useful for decision trees, machine le...

I just came across the `data.tree` #RStats package:

📦: cran.r-project.org/web/packages...
🧑‍💻: github.com/gluc/data.tree
📚: cran.r-project.org/web/packages...

I was looking for a way to create simple tree diagrams of various outcomes and probabilities. Any #dataBS folks have other suggestions?

If you're still following this thread, I highly suggest checking out the full post: www.counting-stuff.com/data-cleanin....

It's full of other great takeaways.

9/9

"Unless you know what people are going to use the data for, you won’t know. Even if you know, you’d still must guess at what is likely to help their analysis and not hurt it. By far the most important thing to do is to fully document every cleaning decision ..."

8/9

I'm a big fan of the practical advice at the end:
* Save a copy of the original data, avoid permanent changes.
* Leave a paper trail.
* Fix what's needed to get your analysis to work.
* Reduce unwanted variation
* Eliminate bias where possible

7/9

Data practitioners are not safe from change, which is the result of operating in environments utilizing modern distributed-systems infrastructure. Thus, cleaning operations will be constantly revisited.

6/9

"The problem with automation is that it can divorce understanding the underlying data from the analysis and interpretation of the data."

5/9

"If you delegate that responsibility someone else [data cleaning], whether it’s another human or a machine, you put yourself at risk of doing something dangerous with your data."

4/9

"The real reason for all this cleaning work is the signal-to-noise ratio in raw data is too poor for purpose we intend. We need to improve data quality to amp the signal for our tools to find."

3/9

"We’re doing cleaning because we want to extract the useful signal from the noise, and we decide certain bits of noise “correctable” at the data point level for that purpose."

2/9

Data Cleaning IS Analysis, Not Grunt Work Also, most data cleaning articles suck

I got introduced to @randyau.com's 'Data Cleaning IS Analysis, Not Grunt Work' post during the #dataBS Conf this week: www.counting-stuff.com/data-cleanin... . I just finished--it was a great read.

Here are some quotes and thoughts I'm walking away with 👇

1/9 #RStats

Probably should have mentioned the date and time:

October 16, 2025, 9am PT / 12pm ET

Just registered for the @rconsortium.bsky.social's 'How to use `pointblank` to understand, validate, and document your data' online workshop. I'm looking forward to learning more from @richmeister.bsky.social.

Sign up here: r-consortium.org/webinars/how...

#RStats #DataBS

Perhaps to give the user the option to write their own custom handling behavior? 🤷‍♂️

The ‘just work without error’ design decisions are something I appreciate from time to time.

I’ll also never stop loving finding those little things base R has already solved. You just have to explore a little bit.

TIL: #Rstats base::system.file() has a `mustWork` argument. It pushes an error if a file is not found. Useful.

I often go beyond 80 characters per line ...

The world continues to turn.

Great resource! 👀
Thanks, @emilhvitfeldt.bsky.social!

ggplot2 4.0.0 A new major version of ggplot2 has been released on CRAN. Find out what is new here.

I am beyond excited to announce that ggplot2 4.0.0 has just landed on CRAN.

It's not every day we have a new major #ggplot2 release but it is a fitting 18 year birthday present for the package.

Get an overview of the release in this blog post and be on the lookout for more in-depth posts #rstats

Clearly I don't know how link shortening works on BlueSky. So, if anyone has trouble opening the above links, just send me a DM and I'll point you in the right direction.

Seeing you have marketing listed in your description
@drsundar.bsky.social
, you might find this dataset useful: developers.google.com/analytics/bi...

I also wrote a blog post on this topic as well. Check it out: www.collinberke.com/blog/posts/2....

BigQuery | AI data platform | Lakehouse | EDW BigQuery is the autonomous data and AI platform, automating the entire data lifecycle so you can go from data to AI to action faster.

Yes, of course! BigQuery is a cloud based data warehouse: cloud.google.com/bigquery. bigrquery is an #rstats package that provides an interface to bridge BigQuery and R (i.e., download data stored in this warehouse). BigQuery has public datasets to experiment with: cloud.google.com/bigquery/pub...

Bqs write by meztez · Pull Request #79 · meztez/bigrquerystorage beta test bq write

+1 for bigrquerystorage to support the BigQueryWrite interface. Keeping an eye on the package's development:

👀: github.com/meztez/bigrq...

I can confirm, data downloads for bigrquery are now blazingly fast 🔥.

#RStats

I've been excitedly waiting for this feature to be released for the #rstats bigrquery package. Now it's finally here! 🏎️💨

Just started seeing this today. Thanks for sharing the fix!

🔗 Links
📝 Notes: www.collinberke.com/til/posts/20...
📦 {gganimate}: gganimate.com
📦 {cfbfastR}: cfbfastr.sportsdataverse.org

I recently enjoyed using the {gganimate} #RStats package for #DataVis animations. So, I explored it further and drafted some notes. What resulted was some example animations using the palmer `penguins` dataset and some B1G QB passing data from {cfbfastR}.

I was pleased with the outcome. Links 👇

Document Listings – Quarto

TIL: #quarto allows you to easily customize the fields used within listing pages: quarto.org/docs/website...

I've been wanting to simplify and make the listings pages on my personal site a little more lightweight: www.collinberke.com

This will be the focus of my weekend 🧑‍💻

#RStats #dataBS