Weston Pace

I'm guilty but I really like this advice. I'm going to switch.

It's pretty awesome to look back and see how far this has come in 10 years!

The Quest for One Million IOPS: Benchmarking Storage at LanceDB Learn how LanceDB benchmarks storage and how we achieved one million disk reads per second.

Nothing more satisfying than hitting those big round number goals: lancedb.com/blog/one-mil...

First is more readable but the second is more authentic.

Work

Reminds me of xkcd.com/1741/

Convolution Peaks

The Gaussian Range

Both software and config can be rolled out gradually with rollback to return to a good state (of course rollback is fallible). I suspect CF does both.

In this case though it sounds like a rollout of service X caused service Y to start failing while service X seemed healthy.

If you enjoy this feeling you should install `sl` from apt.

Pretty sure my stress level can be accurately inferred by the number of unread emails in my inbox

Apache Arrow's Final Frontier: Replacing Outdated Database Drivers A new startup, Columnar, looks to streamline the copying of tabular data across systems, using Apache Arrow and the ADBC API.

A new startup, @columnar.tech, looks to streamline the copying of tabular data across systems, using @arrow.apache.org and the ADBC API.

By @joabj.bsky.social

Async is awesome until it isn't

Also, I guess its "cleaning the filter" not changing it

Maybe only a thing on some washing machines (or only when you have enough pet hair in your home 😅)

My old washing machine had to be taken apart to change the filter but my new one has a little door.

Resetting the garbage disposal with Allen wrench. Changing the filter on clothes washer. Testing and replacing smoke alarms.

Ah, I ran into something very similar yesterday with an async "find or insert" cache. The first caller canceled the request while the insert future was in progress (dropped the future) and that cache key was forever blocked.

Conclusion of a little halloween tradition. If I'm going to traumatize the kids it might as well be interesting.

Announcing Columnar Back to the future of data connectivity

The future of data connectivity is columnar. Today we launched
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇

Nice definition! This matches my use. I also usually have a touch of "please don't hate me I'm doing my best"

A bittersweet story but glad to see a principled stance!

However - a word that exists because someone decided we aren't allowed to start a sentence with "but"

The awkward monkey puppet meme with the text "Well..." from a maintainer of Lance, a lake house format that might just happen to be what the author is describing...

a black and white photo of a woman wearing a turtleneck sweater and a dress . ALT: a black and white photo of a woman wearing a turtleneck sweater and a dress .

Your coworkers about to flood the channel because "I guess he doesn't want threads for this one"

Douglas squirrels are 1/3 the size of gray squirrels but six times more ferocious.

I suspect this will change as caching layers become more mature. The selectivity threshold for cloud storage is something like "one in a million" but more like "one in a thousand" for NVMe.

Also, a self-promotional shout out that you might want to look at lance (lancedb.github.io/lance/format...)

They do a bit of both. The base model is unsupervised and is generally described as "learning the language". The model is then fine tuned with supervision for a specific task.

The "suck up as much data as you can" is for the first part.

Yesterday, OP responded to my 11 year old comment on their 13 year old post with a pedantic correction.

Though I think the "we can't change Parquet" problem is a bit of a false problem. 90% of Parquet users are probably fine to just keep using Parquet. I'm not sure I agree that "the long time archival format" and the "database storage format" need to be the same thing.

That might be next week's blog post ;). Short answer is I see it as a table format problem and not a file format problem. Change "decoder" to "file reader". Change "stored in the page" to "stored in a folder on the table" and change "wasm" to "pluggable" (native or wasm).

Hope this helps, it's fun to see so much exciting innovation in a space that's been relatively quiet for many years!