v

Explicit error - github.com/duckdb/duckd...

Configurable behaviour - github.com/duckdb/duckd...

Partial Recovery - github.com/duckdb/duckd...

Safe truncation - github.com/duckdb/duckd...

I went through DuckDB's WAL, and it does everything I was asking for in my blog post:

1. Per record checksum
2. Explicit error on checksum failure
3. Configurable behavior
4. Partial recovery
5. Safe truncation of the WAL only when WAL contents are checkpointed

paper - ir.cwi.nl/pub/28800/...

A more advanced future direction is self-checking. We have learned to distrust the hardware the database is running on. This is particularly relevant in the edge computing use case, where hardware failures are to be commonplace. One approach is to keep checksums on all persistent and intermediate data and piggy-back checksum verification on scan operators. This might be possible without a significant performance impact. A vectorized engine is particularly suited for this since a chunk of data typically fits in the CPU cache and additional passes are not requiring RAM access

DuckDB on checksums:

Excerpt from interview with Thorsten Ball on tech blogging

Our latest tech blogger interview: @thorstenball.com shares his writing reflections and tips writethatblog.substack.com/p/thorsten-b...

ahh, thanks for the link!

- Expose APIs for metrics (like CPU/RAM usage), profiling, and tracing

🤔

What are some databases that take extensibility seriously and have first-class support? My wishlist:

- Extremely easy to add extensions (including a testing framework)
- Package manager and dependency management
- Versioning
- Sandboxing/isolation
- Hot reload support
- SDKs in multiple languages

My fren joined a new company with a huge Python codebase. I asked him how much of the code is covered by type hints.

“What type hints? More than 50% of the code is still Python 2.” 🥲

Writing High-Performance Clients for Tigerbeetle Insights, updates, and technical deep dives on building a high-performance financial transactions database.

anyways, here is the link: tigerbeetle.com/blog...

I read this article by Rafael Batiati and the performance of Go client with CGO is almost like any other. I know it all depends™️, so I benchmarked my old code, and the numbers do look better.

(It is also totally possible that earlier I didn't benchmark properly)

I always thought Go FFI was slow af. This is a common assumption in the Go community too. A couple of years ago, there used to be so many posts about how slow it is.

Turns out it may not be true and Go might have improved.

never would i have guessed that expecting an error on checksum mismatch was going to be a controversial ask 🥲

link - avi.im/blag/2025/sq...

PSA: SQLite WAL checksums fail silently and may lose data This is a follow-up post to my PSA: SQLite does not do checksums and PSA: Most databases do not do checksums by default. In the previous posts I mentioned that SQLite does not do checksums by default, but it has checksums in WAL mode. However, on checksum errors, instead of raising error, it drops all the subsequent frames. Even if they are not corrupt. This is not a bug; it’s intentional.

Published a new post: "PSA: SQLite WAL checksums fail silently and may lose data"

This is a follow up to my previous posts. When SQLite encounters checksum failures in WAL, instead of raising an error, it drops all subsequent frames; even if they are not corrupt. It's not a bug

cacm.acm.org/practice/sys...

I liked how this article lists various methods and how they're used to solve particular cases. Also includes references to some great papers.

Also, TIL that AWS has a fault injection service where customers can inject simulated faults from API errors to I/O pauses and failed instances.

Loved reading this excellent post by Marc Brooker and Ankush Desai: Systems Correctness Practices at Amazon Web Services on how they use formal methods internally at AWS. One of the best articles I've read this year on formal verification and testing.

Rickrolling Turso DB (SQLite rewrite in Rust) This is a beginner’s guide to hacking into Turso DB (formerly known as Limbo), the SQLite rewrite in Rust. In this short post, I will explore how to get familiar with Turso’s codebase, tooling, and tests (hereafter mentioned as just Turso). Disclosure: I work in the same organization where Turso is being developed. However, I am not a contributor (yet) and most of my work is on the Turso Server. I don’t contribute to the Turso Database, but I am somewhat familiar with SQLite. I wanted to take Turso for a spin and explore the codebase. I timeboxed my experiment for 6 hours, including writing this blog post. Do note that Turso is under heavy development, so much so that core developers spend more time resolving merge conflicts than writing code. So expect most of the content in this blog post to get obsoleted by next month.

Published a new blog post: Rickrolling Turso DB

avi.im/blag/2025/ri...

source - tangled.sh/@rockorager....

post - rockorager.dev/log/lsr-ls-b...

Syscalls Data gathered with strace -c on a directory of n plain files. (Lower is better)

I found out about lsr, an ls rewrite in Zig using io_uring by @rockorager.dev, started as a fun experiment to learn io_uring. It uses 35x fewer sys calls!

I loved reading the companion blog post too.

Rust is a great language. I really enjoy using it. However, when it comes to database systems, I would still choose C++ over Rust. I know this may be unpopular, but during the development of CedarDB, we discussed this and decided to continue using C++. The main reasons were safety and undefined behavior (UB). Shouldn't safety be a reason to choose Rust over C++? In theory, yes. That's what I personally enjoy about Rust: you get much more safety by default than in C++. However, when you absolutely need to use potentially unsafe code, Rust makes things much harder. In a code-generating database system like CedarDB, which uses many low-level CPU features, intrusive data structures, and optimistic, racy memory access with validation, you would definitely need the unsafe keyword. Once you write unsafe code in Rust, all bets are off. It's very easy to run into UB with unsafe code, much easier than with C++. The rules of what constitutes UB are already complex in C++ (I would know, I have

CedarDB on why they picked C++ over Rust

The super high-level answer is that in the context of TigerBeetle the tradeoffs weighed in favor of Zig. I wrote a bit about the relevant context here: https://matklad.github.io/2023/03/26/zig-and-rust.html The main benefit Rust provides is memory safety through composable abstractions. Due to TigerBeetle’s architecture (no dynamic resource allocation, single threaded control loop, tight direct integration between network/store/state transition rules, simulation-based testing), the magnitude of this benefit is relatively smaller than it usually is. OTOH, the benefits of Zig loom larger: simple language ability to both use standard hash map and at the same time statically guarantee that no allocation happens after startup more expressiveness/less noise for some lower-level things (pointers that carry alignment in the type, mem.asBytes, some more advanced meta programming)

matklad on why TigerBeetle picked Zig

linked post - matklad.github.io/2023/03/26/z...

fstrings.wtf - Python F-String Quiz Test your knowledge of Python's f-string formatting with this interactive quiz. How well do you know Python's string formatting quirks?

How well do you know Python’s f-strings? I built a thing inspired by JavaScript’s Date constructor quiz but for Python. fstrings.wtf

I don't have more information, but this comment by Marc Brooker provides more context

lobste.rs/s/cbd7rn/jus...

Just make it scale: An Aurora DSQL story AWS Senior Principal Engineers, Niko Matsakis and Marc Bowes, take us inside Aurora DSQL's development: scaling write operations without two-phase commit, overcoming garbage collection hurdles, and em...

okay, not AWS, but Werner

www.allthingsdistributed.com/2025/05/just...

The entire post is full of praise for Rust and makes an excellent case for why Rust is a great choice for writing a new database. Go read it! 4/5

They wanted to extend Postgres. Instead of C, they chose Rust again for memory safety reasons. For tooling reasons, they rewrote the control panel, and well, you guessed it right, in Rust. 3/5

The code was 10x faster even though they didn't make any attempt to optimize it. They had spent years carefully tuning the Kotlin version which did 3,000 TPS. The Rust one clocked 30,000 TPS. Insane! 2/5

AWS published a blog post: An Aurora DSQL Story. It's less about database development and more like a love letter to Rust.

DSQL's initial development was in Kotlin, but they weren't happy with its performance. So two engineers, who had never written Rust before, rewrote a component in Rust. 1/5