Nikhil Benesch

They’re calling it our most boring feature yet.

boring(n): mundane; works exactly as expected; the highest praise you can give a database

tpuf Python client went async today! 🐡 ❤️ 🐍

Async client perf slightly edges out sync client perf under heavy query load.

github.com/turbopuffer/...

Big day today at turbopuffer HQ.

when we said "coming soon" we really meant it

now puffin' in aws-us-east-1 and aws-eu-central-1

I did! Couldn't pass up the opportunity to work with this team!

Things move fast at turbopuffer. Now puffin' in aws/us-east-1 and aws/eu-central-1 too.

I've been dreaming about conditional writes on S3 for years. I couldn't have asked for a better way to celebrate than getting to ship tpuf on AWS. 🐡

Try C4A, the first Google Axion Processor | Google Cloud Blog The custom Arm-based processor is designed for general-purpose workloads like web and app servers, databases, analytics, CPU-based AI, and more.

C4A: cloud.google.com/blog/product...

It’s a rare day that my love of going to battle with build systems pays off like this. Kudos to GCP for a very impressive new SKU. 🐡💨

Couldn't be more excited to be joining the team at @turbopuffer.bsky.social. 🐡💨

Just catching up on my NULL BITMAPS and this is easily the best intuition for write skew I've ever seen described. The analogy to merge skew in a codebase is genius.

Well well well: www.crunchydata.com/blog/pg_incr...

Incremental pipelines come to Postgres via Crunchy Data! This is like "dbt incremental", not true incremental view maintenance like @materialize.com or Snowflake's dynamic tables, but it's a neat step towards IVM.

Of course! Will be very excited to give the new DynamoDB-free SlateDB a spin.

The 6th (and last of 2024!) NYC Systems talks are next Thursday! We've got @jaronoff.com of Omlet and @benesch.bsky.social of Materialize speaking. :)

nycsystems.xyz/december-202...

Same! I’ve been using Amethyst for over ten years at this point. When I discovered the project in 2013 I never expected that ianyh would still be lovingly maintaining it a decade later. One of my most loved pieces of software, for sure.

tl;dr the 10x TPS claim actually *is* based on a small but novel optimization inside of S3! Table buckets understand Iceberg naming conventions and adjust S3's rate limiting so that you start with budget for 55k/35k GETs/PUTs per sec rather than the general purpose default of 5.5k/3.5k.

YouTube video by AWS Events AWS re:Invent 2024 - [NEW LAUNCH] Store tabular data at scale with Amazon S3 Tables (STG367-NEW)

It just occurred to me that while the 3x query performance claim is based on a comparison to uncomplicated tables, I hadn't actually seen the evidence for the 10x TPS claim.

After some digging I just found the explanation in the re:Invent talk (start at 14:24): youtu.be/1U7yX4HTLCI?...

Exciting! I'm so here for it. 👀

Disclaimer: I'm not involved with DuckDB, ClickHouse, or Iceberg, so take this all with a grain of salt.

More Than 2x Faster Hashing in ClickHouse Using Rust Rust’s rich type system and ownership model guarantee memory-safety and thread-safety. There are a fairly large number of useful libraries written on it, so we considered using them in ClickHouse.

Rust might be feasible! In fact I think DuckDB's Delta extension is based on a Rust library. And ClickHouse is starting to integrate some Rust too: clickhouse.com/blog/more-th...

Sadly Rust's Iceberg library is still relatively immature (e.g. no support for writes: github.com/apache/icebe...).

Lies we tell ourselves to keep using Golang In the two years since I've posted I want off Mr Golang's Wild Ride, it's made the rounds time and time again, on Reddit, on Lobste.rs, on HackerNews, and elsewhere. And every time, it elicits the ...

Calling Go code from another language is hard enough that' it's rarely done—especially in high performance contexts. (🌶️ alert, but fasterthanli.me/articles/lie... covers this—see "the only good boundary with Go is a network boundary.")

Nice, looking forward to it!

Friendlier SQL with DuckDB DuckDB offers several extensions to the SQL syntax. For a full list of these features, see the Friendly SQL documentation page.

I’m not 100% sure on the chronology but I believe DuckDB originated many of these pragmatic SQL UX improvements: duckdb.org/2022/05/04/f...

GitHub - apache/iceberg-cpp: Apache Iceberg C++ Apache Iceberg C++. Contribute to apache/iceberg-cpp development by creating an account on GitHub.

Keep an eye on github.com/apache/icebe...! AFAIU the lack of a C++ Iceberg library is what’s been holding back full Iceberg support in DuckDB and ClickHouse.

Databases Are Commodities. Now What? How will vendors differentiate when databases are commoditized? I've got three ideas.

@chris.blue you might be interested in this. Something like this fits in very neatly to your views on the commoditization of the PostgreSQL dialect/protocol (materializedview.io/p/databases-...).

Indeed. That's one of the more cursed stories I've heard. Might be worth a uv bug report, if the constraints are indeed solvable!

Have you tried `uv` yet? docs.astral.sh/uv/

It's night and day compared to pip.

👋🏽 Well hello, Spiral.

(for posterity for anyone else following along: it seems to be sufficient to define the external consistency guarantee in terms of T1's commit timestamp and T2's read timestamp because SI already implies that a txn can't commit before it starts: bsky.app/profile/bene...)

This is so cool to see!

Out of curiosity, which sort of features were hard to test via SQL and/or PL/pgSQL? I would have naively assumed that all observable behavior of PostgreSQL was observable via the SQL interface. 😅