Dan Forsberg

Secure Remote Secrets Storage - made it too secret..

GitHub - dforsber/boilstream-extension: Secure Remote Secrets Storage for DuckDB Secure Remote Secrets Storage for DuckDB. Contribute to dforsber/boilstream-extension development by creating an account on GitHub.

DuckDB Secure Remote Storage community extension!

- Now with session resumption!

#duckdb #opaque #pake #boilstream

github.com/dforsber/boi...

GitHub - boilingdata/boilstream: Diskless high-throughput data ingestion (Kafka, Flight) server with local DuckDB caching. Writes optimised Parquet files onto Object Storages. Diskless high-throughput data ingestion (Kafka, Flight) server with local DuckDB caching. Writes optimised Parquet files onto Object Storages. - boilingdata/boilstream

🚀 Scalable Browser click/event streams to BoilStream!

github.com/boilingdata/...

#duckdb #kafka #arrow #flightsql #postgres #rust | Dan Forsberg 🏠 BoilStream Technical Architecture! ⁉️ 📋 🧠 Here is technical architecture of BoilStream - A DuckDB Server with Steroids! 🤗 We have been playing with Candle AI/ML pipelines as well, but it's not y...

🏠 BoilStream Tech Architecture! ⁉️

On my post at LinkedIn:
www.linkedin.com/posts/danfor...

GitHub - boilingdata/boilstream: Diskless high-throughput data ingestion (Kafka, Flight) server with local DuckDB caching. Writes optimised Parquet files onto Object Storages. Diskless high-throughput data ingestion (Kafka, Flight) server with local DuckDB caching. Writes optimised Parquet files onto Object Storages. - boilingdata/boilstream

💯 👉🏻 Power BI now can read BoilStream v0.7.10 types!

github.com/boilingdata/...

BoilStream - A Streaming LakeHouse | Dan Forsberg 🎉 🧠 Kafka interface to DuckDB! What do you say Stanislav Kozlovski, why not Kafka directly into DWH? 😄 🚀 BoilStream v0.7.7 introduces initial version of high performance Kafka interface! 🚕 It runs...

⁉️ First in the world? Kafka interface to DuckDB! 🦆

www.linkedin.com/posts/danfor...

Actually, I also downloaded Tableou and needed to make one small fix and it worked with all types out of the box. No type downgrading needed with Tableou 👏

BoilStream - A DuckDB Server written in Rust. Download small binary and run - connect with any BI Tool or postgres cli.

GitHub - boilingdata/boilstream: Diskless high-throughput data ingestion server with local DuckDB caching. Writes optimised Parquet files onto Object Storages. Diskless high-throughput data ingestion server with local DuckDB caching. Writes optimised Parquet files onto Object Storages. - boilingdata/boilstream

🚀🎉 BoilStream v0.7.5 out!

Power BI now works with all data types!
See the demo_database.sql test suite on GitHub.

NOTE: Power BI uses old version of npgsql, so we had to downgrade some temporal (TEXT) and list types (JSON). But only for Power BI clients.

github.com/boilingdata/...

🚀 Run MS Power BI withour DuckDB Server!

🦀 We added extensive Postgres compatible interface and put it around DuckDB as a server with Rust. You can use psql, bgbench, DBeaver, Grafana, Superset, ..and Power BI.

#powerbi #duckdb #analytics

Download for free!
boilstream.com

🚀 BoilStream is a DuckDB server with Postgres interface for BI Tools. For data ingestion, just use DuckDB Airport community extension and you get GBs/s data ingestion speed to DuckDB database while running realtime analytics over the postgres interface.

github.com/boilingdata/...

🦆 How fast can @DuckDB ingest data? 🚄

It's much more than I thought. You can read about it on BoilStream website and Documentation.

🚀 We have new BoilStream release which lands data on local DuckDB db AND on S3 Parquet at the same time.

www.boilstream.com

💡 You can now have multiple storage backends with BoilStream, like S3 and Filesystem. Or two different S3 buckets, or Minio S3 and AWS S3, or..

www.boilstream.com

🚀 Skip the Pipeline. Stream to Gold. 🏆

You can write CREATE VIEW from DuckDB client against our Stream Processor server (DuckDB Airport extension) and it forks the base topic into derived topic(s) for you while it runs zero-copy SQL over the incoming data stream in real-time.

www.boilstream.com

Add support for Airport CREATE VIEW statement by dforsber · Pull Request #20 · Query-farm/airport Support for CREATE VIEW This PR adds support for CREATE VIEW statements on the Airport extension. Motivation I would like to add support for materialised views on my FlightRPC Data Ingestion Server...

⫚ SQL stream processing with zero-copy over Arrow IPC data with DuckDB 🦆 🏹

github.com/Query-farm/a...

🔐 BoilStream now supports all major SSO providers 🚀

The paid version includes support for TLS and Bearer token authentication for all major SSO providers: Azure AD (Entra ID), AWS Cognito, GCP, Auth0, and Okta. And includes 3-level authorization (groups, scopes, id).

www.boilstream.com

🚀 Launching boilstream.com - Making streaming data dead simple. Directly from DuckDB into Parquet files on S3 from thousands of concurrent writers and GBs/s with single node. 🔥

See DEMO on the web page.

"single thread server". Had to use a fair amount of threads to saturate it..

70k RPS with 1KB packets with single thread on arm64 macbook pro M3 (Nov 2023). Not bad.

GitHub - boilingdata/mcp-server-and-gw: An MCP stdio to HTTP SSE transport gateway with example server and MCP client An MCP stdio to HTTP SSE transport gateway with example server and MCP client - boilingdata/mcp-server-and-gw

Update README! Example MCP server.

Connect Claude Desktop to your MCP compatible Server.

github.com/boilingdata/...

Cool, to be able to do meaningful changes, you need to run excessive big tests to see clearly and above noise. It is not about whether big data exists or not in this case. Congrats!

The total time to perform a cold run of the TPC-H queries on SF300 shows an almost 3× speedup between DuckDB 1.0.0 and DuckDB 1.2.0. The benchmark was run on a MacBook Pro (M1 Max / 64 GB RAM).

We have implemented the basic data analytics requirement 😍. We can reliably consume Postgres CDC stream with ephemeral node, including DR without full resync needs. The system is transaction safe with Icebreg Table on S3 target and can even detect if Postgres was restored from a backup. Interested?

Have been running Claude Code (agent based) and like it a lot. You still need to keep control and verify direction and quality.

YouTube video by BoilingData 2025 02 13 Snowflake Integration

❄️ Snowflake integration into Boiling Insights. Copy tables efficiently with optional sampling to local DuckDB!

Copied tables are automatically attached and detached on delete. Easy to run queries..

#snowflake #duckdb #boilinginsights

youtu.be/GisDleU8PDE

🚀 New video coming tomorrow about ❄️ Snowflake integration into Boiling Insights, and table sync to DuckDB db.. the table jobs run on bg using an optimal DAG executor and the GUI shows progress.

YouTube video by BoilingData Introducing Boiling Insights Remote Backends

Small Remote Backends demo.

youtu.be/m3DV3GlDu-M

GitHub - dforsber/duckdb-kton-extension: DuckDB KTON extension DuckDB KTON extension. Contribute to dforsber/duckdb-kton-extension development by creating an account on GitHub.

Zig is amazing as is DuckDB!!
Created for fun and as an example fixed column text file scanner extension for DuckDB (DanskeBank KTON file). By no means production proof and it actually misses couple of columns in the end. But this was less than an evening work.
github.com/dforsber/duc...

We are happy to announce DuckDB v1.2.0 “Histrionicus”!

The new release has several usability, security and performance improvements. It has new features for the CSV and Parquet formats, as well new (opt-in) options for DuckDB's own format.

Read more at duckdb.org/2025/02/05/a...

🎉 New Boiling Insights version available along with support for Remote Backends!

From single-user local to multi-user distributed/centralised!

Beefed up DuckDB hosted experience with integrations to AWS DynamoDB and Glue Catalog.

Application and Backend container. Simple.
www.boilinginsights.com

What is the current best practice lib(s) to establish secure datagram connection from browser to nodejs based server?