Peter Boncz

@duckdb.org 1.4.0 is feature-packed: MERGE INTO, compressed in-mem DBs, Iceberg writes..

PhD students also contributed:
- Laurens Kuiper: new k-way parallel mergesort duckdb.org/2025/09/24/sorting-again.html
- Lotte Felius @ccfelius.bsky.social: on-disk DB encryption
- Denis Hirn: materialized CTEs

Laurens Kuiper from @duckdb.org presented DuckDB's new memory assignment policy to run multi-join pipelines out-of-core with gracious performance degradation when join hash tables increasingly do not fit in RAM.

A well-attended and -delivered talk!

paper: vldb.org/pvldb/vol18/p2748-kuiper.pdf

Tobias Schmidt (TUM) @vldb.bsky.social at VLDB2025 presented SQLStorm, which uses LLMs to generate a huge amount of complex queries

SQLStorm now has 18K different complex queries and runs on a large real-world dataset (stackoverflow)

paper: vldb.org/pvldb/vol18/...
code: github.com/SQL-Storm/SQ...

Very honored to receive the @vldb.bsky.social 2025 Test of Time Award for the Join Order Benchmark (JOB)

Kudos to my very talented TUM co-authors, specifically Viktor Leis who was the driving force & gave a great award talk.

paper: www.vldb.org/pvldb/vol18/p5531-viktor.pdf
JOB: event.cwi.nl/da/job

Azim Afroozeh gave a great talk at @vldb.bsky.social VLDB2025 in London on the FastLanes file format.

FastLanes compresses 1.4x better than Parquet/snappy and allows 40x faster reads on the PublicBI dataset!

Paper: vldb.org/pvldb/vol18/p4629-afroozeh.pdf
Code: github.com/cwida/FastLanes

@sigmod2025.bsky.social Berlin is a wrap. Many 🙏 to the organizers!

Next stop is @vldb.bsky.social London to present
- github.com/cwida/FastLanes v0.1 of a new big data format
- spilling multi-operator joins (via @duckdb.org)
- the SQLStorm benchmark of 30k LLM-generated complex queries (via TUM)

Some pics of Leonardo Kuffo presenting his SIGMOD2025 paper on PDX.

PDX is a vertical layout that can accelerate vector search in principle in any vector index technique (it makes the distance calculation faster, using better SIMD + pruning).

ir.cwi.nl/pub/35044/3504…
github.com/cwida/PDX

And.. Azim Afroozeh put a lot of effort in open-sourcing the ALP floating point compressor (github.com/cwida/ALP). Leonardo Kuffo had written with him the SIGMOD2024 paper which now won a reproducibility award!

+ 🙏🙏 to the reproducibility and artifacts committee - this is a ton of work.

But @cwi_da has no reason to complain, here in Berlin.

Leonardo Kuffo at the preceding DaMoN2025 workshop won the Best Paper Award for a study that showed that for vector databases, it matters a lot which AWS CPU you pick.

Congratulations to him!

SIGMOD2025 for the 1st time used a schedule where most papers are presented as posters only.

Tips for next time:
- gather user interest data prior to deciding poster-or-paper & room assignment.
- present posters in a (high ceiling) room with good acoustics & allot enough presentation space + time.

🎞️ The stream of “DuckLake & The Future of Open Table Formats”, a conversation between Hannes Mühleisen and Jordan Tigani, will start in two hours!

The opening talk of #systemsdistributed, organized by our friends @tigerbeetle.com in the Eye Film museum in Amsterdam, was given by @hannes.muehleisen.org of
@duckdb.org about:

DuckLake (ducklake.select)

and this was very well received

movie poster refers back to CIDR2025 😄

DuckLake: leverage DB tech for Data Lake metadata.

works on duckdb, postgres, MySQL & SQLite

provides:
- multi-statement &
multi-table transactions
- SQL views
- delta queries
- encryption
- low latency: no S3 metadata &
inlining: store small inserts in-catalog
and more!

A new preprint from database researchers found DuckDB the most environmentally efficient system: arxiv.org/pdf/2504.18980

Introducing the DuckDB Local UI, the easiest way to explore local data files with DuckDB. Built in close partnership with @duckdb.org for the community.

duckdb -ui

Learn more:
duckdb.org/2025/03/12/d...

We strive to ensure that the DuckDB project stays open-source in the long term. That's why we set up the non-profit DuckDB Foundation in 2021. The Foundation owns the intellectual property of the DuckDB project and enshrines the availability of DuckDB as open-source in its notarized statutes.

Vanessa Evers appointed new director of CWI The Board of NWO-I, the institute organization of NWO, appointed Prof. Vanessa Evers as Director of CWI, the national research institute for mathematics and computer science in the Netherlands. In mid...

Vanessa Evers will be our new director!
Excited we will be able to draw on her expertise in integrating AI&tech in social interactions.
More than ever, it is important technology is advanced in ways it positively contributes to society.
www.cwi.nl/en/news/vanessa-evers-appointed-new-director-of-cwi

CIDR2025 is a wrap!

Lived the many interesting papers & discussions, Gong Show, @duckdb reception..

ACM president Yannis Ioannidis gave an inspiring talk on open science.

Proceedings are in ACM DL & VLDB (see cidrdb.org).

🙏 all in+outside @cwi-amsterdam.bsky.social who helped organize!!

In five days the CIDR2025 conference (cidrdb.org) will start, and we are expecting around 170 attendees from all over the world.

On an unrelated note, the exotic "goldeneye" duck was just spotted in The Netherlands!

See: bit.ly/duck-goldeneye

YouTube video by LDBC Linked Data Benchmark Council DuckPGQ: SQL/PGQ in DuckDB

My @ldbcouncil.org TUC talk is online! 🎥 Learn about #DuckPGQ and #SQL/#PGQ here:
👉 www.youtube.com/watch?v=Fzci...

Catch me at @fosdem.bsky.social on Feb 1 in the Data Analytics room, where I’ll continue spreading the word about #DuckPGQ and #SQL/#PGQ. Hope to see you there! 🚀 #FOSDEM2025

Many congrats @hannes.muehleisen.org!

A well-deserved award, recognizing the innovations in @duckdb.org - the most successful open-source DB system to come from @cwi-amsterdam.bsky.social

Let me also honor his 1st PhD student, Mark Raasveldt (DuckDB Labs CTO), instrumental in shaping the project.

@andypavlo.bsky.social's yearly database in review is out, and fun to read. Mentions @duckdb.org in the context of new postgres integrations ("shotgun weddings"?).

Andy will again be in Amsterdam for CIDR2025 (Jan 19-22) & there are 4 days to register for it: cidrdb.org/cidr2025/registration.html

CIDR2025 will once again be held in the Mövenpick hotel at the waterfront in the city center of Amsterdam, a walkable distance from the Central Station (just take a train there from the airport, no need for a taxi/uber).

Amsterdam is once again hosting my favorite event, the Conference on Innovative Data Systems (CIDR2025).

Check its exciting program: www.cidrdb.org/cidr2025/pro...

It will be held January 19-22 in the Amsterdam Mövenpick hotel.

Plan your trip quickly, because registration closes on Thursday!

duckpgq DuckDB Community Extensions Extension that adds support for SQL/PGQ and graph algorithms

Exciting milestone: The DuckPGQ extension for #DuckDB has surpassed 10,000 downloads!🎉

A huge thanks to the community for supporting DuckPGQ for graph analytics. Stay tuned—the next update will bring property graph creation over attached databases!

Explore DuckPGQ here: duckdb.org/community_ex...

Promotional image for DuckCon #6 in Amsterdam, taking place on January 31, 2025, at Pakhuis de Zwijger. The text highlights the talk topic: ‘Unlocking graph analytics in DuckDB with SQL/PGQ,’ accompanied by a headshot of the speaker, Daniel ten Wolde, and the DuckDB Foundation logo.

Excited to speak at #DuckCon #6 in Amsterdam on Jan 31, 2025!🎉

I’ll share how #DuckDB unlocks graph analytics with SQL/PGQ from the SQL:2023 standard using the #DuckPGQ extension.

📍 Free to attend & livestreamed on YouTube!
📅 Details + register: duckdb.org/events/2025/...

🚀 Hope to see you there!

MotherDuck team photographed in The Netherlands

Happy to see @motherduck.com opening shop in my hometown Amsterdam: bit.ly/motherduck-a...

In reality, they have already been renting offices for 1.5 years close to the Database Architectures research group at CWI, but with a Dutch legal entity, and soon an own office, things are solidifying.

Thanks Andy. That is a big honor and it was an honor to have you in Amsterdam. Enjoy the thxgiving break!

60fps of UX Joy with DuckDB+CloudBoaz by Boaz Leskes (DBDBD 2024) The Dutch-Belgian DataBase Day (DBDBD) is a yearly one-day workshop, organized in a Belgian or Dutch university, whose general topic is database research. DBDBD 2024 will be held at Science Park in Amsterdam, The Netherlands. Website: https://cwida.github.io/dbdbd2024/ In the age of ever more powerful hardware, where your laptop can do more than your typical Datacenter server, MotherDuck leverages DuckDB’s state of the art analytical prowess to drive compute down to your laptop as well as making the most of the Cloud. Combine DuckDB’s versatility to run everywhere (including your browser), augment it with a server-less CDW, and you get (interactive) analytical sessions delivering results in unprecedented speed. So fast it updates your dashboard in 60fps. Biography: Boaz Leskes (MotherDuck Amsterdam) is part of MotherDuck’s founding team and leads its database group. In past life, he spent some years on distributed systems, (Elastic)search and cloud platforms. Will happily talk to any of these, or speed skating, kite surfing, rowing, or any other thing of interest.

The closing of DBDBD 2024 was the "Amsterdam Data Systems" session, with talks from @databricks.bsky.social,
@motherduck.com and @clickhouse.com (due to illness,
@weaviate.bsky.social could not make it). These companies all have a significant presence in Amsterdam.

Videos: bit.ly/cwida-ams-da...

But there's more! Last Friday, the day after the Dijkstra Award, it was our honor to host the 20th Dutch-Belgian Database Day (DBDBD2024) at CWI.

It was well-attended and lively. 🙏 to all who made it possible!

DBDBD2024 pictures are on cwida.github.io/dbdbd2024 (scroll down):

Dijkstra Fellowship Acceptance Speech by Marcin Zukowski (Dijkstra Award 2024) Website: https://www.cwi.nl/en/events/dijkstra-awards/cwi-lectures-dijkstra-fellowship/ About the Dijkstra Fellowship The Dijkstra Fellowship is named after former CWI researcher Edsger W. Dijkstra, who was one of the most influential scientists in the history of CWI. Dijkstra developed the shortest path algorithm, among other contributions. The first Dijkstra Fellowships were awarded to David Chaum and Guido van Rossum in 2019. Dijkstra Fellowship 2024 for Marcin Żukowski Marcin Żukowski started his career at CWI. He did his MSc and PhD research on database management system architectures in our Database Architectures (DA) group. As a PhD student under the supervision of Peter Boncz, he developed the innovative concept of vectorized execution to improve the performance of database queries. This research received the DaMoN 2007 Best Paper Award and also the CIDR 2024 Test of Time Award, established by the Conference on Innovative Data Systems Research (CIDR). After his PhD, Żukowski co-founded CWI spin-off VectorWise (now Actian), turning his research into a high performance and highly scalable analytical database system. It became the blueprint for analytical databases, that is still widely used. After yielding a rapid technological and commercial growth, he left the company in 2012 to co-found Snowflake in Silicon Valley. Snowflake offered the first cloud-based data warehousing service that is truly designed for the cloud. Notable features are that it is an ‘elastically’ growing and shrinking system based on how busy it is, separating computation from storage, and automating many administration and configuration tasks. Snowflake uses vectorized query execution and lightweight compression methods in its columnar data storage, two techniques that were co-designed by Żukowski during his PhD years at CWI. Role model After leaving Snowflake earlier this year, Marcin Żukowski stays connected with academia by supervising students, publishing papers and taking part in computer science events. He is also an investor and advisor, supporting technology development and innovation in his home country Poland. “Marcin is an excellent example of how to apply CWI's mission in practice. He used his PhD research at CWI to create versatile foundational software products that are now widely used, and shares his knowledge and experience with the public and in particular with young technology entrepreneurs”, CWI director Ton de Kok says. CWI Lectures combined with Dijkstra Fellowship award Topics of the CWI lectures are related to the architecture of data processing and analysis systems.

Furthermore, we have now posted the videos of these lectures in the Dutch Seminar on Data Systems Design (DSDSD) YouTube channel:

bit.ly/cwida-dijkst...

many 🙏 to Daniel ten Wolde & Leonardo Kuffo Rivero for editing these!