Kafka Streams

While we are waiting for Apache Kafka 4.1.0 release, the development does not stop. We just completed KIP-1034 (cwiki.apache.org/confluence/d...) which adds a DLQ feature to Kafka Streams. Very exciting!

Next at #Current25 a panel discussion on the past, present and future of @kafkastreams.bsky.social. I love the background...the likeness is uncanny...

Apache Kafka 3.9.1 is now available for download: buff.ly/5ONjJgB

This bug-fix release has several fixes, and most importantly, it adds Java 23 support. #opensource

There is a few more others which are more minor. So we stick with these highlights for now. If you want to know them all (also for older releases), check out the wiki: cwiki.apache.org/confluence/d...

That’s a wrap!

\TheEnd

KIP-1112: cwiki.apache.org/confluence/d...

Injecting custom boiler plate code into all Processors is now easily possible, via this new “processor wrapper” concept. Useful for PAPI users, but invaluable for the DSL!

\6

KIP-1106: cwiki.apache.org/confluence/d...

This is a shared KIP for the consumer and Kafka Streams. A new auto.offset.reset policy “by duration” is added. Very useful for tiered storage setups when you don’t want to use “earliest” but still want to start at some point in the past.

\5

KIP-1104: cwiki.apache.org/confluence/d...

Foreign-key table-table joins are very popular. However, the FK could only be extracted from the value so far. This unnecessary limitation is gone now, by passing in both key and value into the FK-key extractor.

\4

KIP-1091: cwiki.apache.org/confluence/d...

Let’s make KIP-1076 even more useful. A few key metrics like client-state and thread-state are Strings and thus cannot be collected via KIP-714. This KIP adds numerical twins for these metrics to make them collectible!

\3

KIP-1076: cwiki.apache.org/confluence/d...

This is an extension to the famous KIP-714. We now allow to register user metrics for broker side collection. Kafka Streams uses these new APIs to register its own metrics, so all Kafka Streams metrics can now be collected broker side.

\2

Apache Kafka 4.0 was released recently. We got a couple of a cool KIPs shipped.

KIP-1065: cwiki.apache.org/confluence/p...

The ProductionExceptionHandler becomes more flexible. It’s now also called for retriable errors. This allows you to break (previously infinite) retry loops.

\1

Kafka Streams 101: Optimizing your apps for maximum performance Learn the concepts, configs, and application design strategies to get your Kafka Streams apps running as optimally as possible.

Next up in the Kafka Streams 101 series: performance optimization!

Admittedly this one goes way beyond just the "101" level, with general tips to incredibly specific optimization techniques.

Check it out to get a handle on your application's performance: bit.ly/43zXoQM

@intellijidea.com's IDE has a great Apache Kafka plugin.

This is a @kafkastreams.bsky.social's session window changelog topic -- maybe I can figure out how to get support natively for Kafka Stream windowing keys.

Writing and a running a Kafka Streams app can be trickier than you'd think! We have some basic examples in the repo but they don't cover everything. Glad to have more material focused on helping new people get into Kafka Streams!

Kafka Streams 101: The Most Important Configs Learn which configurations are most critical for a reliable and efficient deployment

We're keeping the ball rolling on the Kafka Streams 101 series with yet another blog post! This time the focus is on configuration, and breaking down the most important configs to focus on for various needs (such as correctness, resiliency, etc)

Check it out here: www.responsive.dev/blog/importa...

Kafka Streams 101: Topology Upgrades A Kafka Streams 101 lesson to learn how to identify unsafe changes to a topology and the techniques to mitigate them

The 2025 blog post series has begun and we're kicking things off with the "official" Kafka Streams upgrade-guide -- check it out for best practices when setting up a new app, rules for safe vs unsafe upgrades, and tips for making the process go smoothly: www.responsive.dev/blog/topolog...

#StreamProcessing with @kafkastreams.bsky.social

Another holiday 🎁 delivered. We completed issues.apache.org/jira/browse/... to remove all deprecated #KafkaStreams APIs for the upcoming #ApacheKafka 4.0 release!

It was an outstanding community effort and huge win!

We merged 36 PRs from 14 different contributors, removing 13K LOC.

\1

Note: this is the official Apache Kafka Streams account and does not endorse specific vendors, though we may share content from external sources that is relevant to open source Kafka Streams.

For hot takes and broader community updates, follow the unofficial account @thestreamingotter.bsky.social

Kafka Streams has officially come to Bluesky! Follow this account for updates on everything Kafka Streams and keep up to date with new features, bug fixes, and upcoming KIPs.

Long live the otter!