Pretty relevant these days

Earlier this week, I presented a talk at the Vancouver Systems Meetup: "Streamling: Building a Lightweight Data Streaming Runtime on Apache DataFusion".

I'm excited to announce Streamling: lightweight, performant and extensible data streaming runtime. Streamling is built using Rust, Apache Arrow and Apache DataFusion.

Analyzing a 1000+ line Flink SQL pipeline today 🫠

Found a love letter to @andypavlo.bsky.social at #PgConf in Vancouver

Just in the nick of time, it's my round-up of Interesting Links for April 2026! There's links about all sorts of stuff around Data Engineering, CDC, Iceberg, Data architectures, and even a little hint of AI (but only the useful kind, not the gross hype-y kind). rmoff.net/2026/04/30/i...

Last night 🤟🥁

Sharing some thoughts this morning about different paths you can take as a software engineer who's looking for more independence, and perhaps, eventually becoming a solopreneur. It's not just about building a product or becoming a consultant. www.independentengineer.co/p/software-e...

Data Streaming Academy has tutorials now! The first tutorial covers reading and writing Kafka consumer offsets in Apache Flink using the State Processor API. streamacademy.io/tutorial/fli...

Recently, I spent a lot of time benchmarking various Iceberg sinks (Spark, Flink, Kafka Connect) and trying to beat @supermetal-inc.bsky.social for moving data from Postgres to Iceberg. Here's my report: thenewstack.io/postgres-ice...

Is this peak bubble? techcrunch.com/2026/04/15/a...

Never gets old

Second post in The Independent Engineer (my new newsletter!) focuses on a very important question: how to (re)discover your calling as a software engineer? I believe specialization is important, especially if you'd like to explore the idea of becoming a solopreneur in the future.

Why does it take AWS 40 (!) minutes to provision a 3-node MSK cluster? Do they manually bring a new server rack for this? I understand why a rolling update might be slow, but creating from scratch? Really?

I'm starting a new newsletter! For a while, I wanted to share my experience building an independent business. Specifically, building a business as a software engineer. A business where you keep building systems, solving problems, writing code - whatever you really enjoy doing.

It's beautiful github.com/nicolasmd87/...

The Advanced Apache Flink training is now available as a self-paced, on-demand course! I ran the first cohort of the training in January and also had the chance to conduct a few corporate training sessions with the same program. I’ve received a lot of great feedback, iterated and polished content.

Recording and slides available from the DataFusion Stockholm Meetup: Recording: youtu.be/9u4cNmL14Xs Slides: github.com/apache/dataf...

I'm not asking much, can we open two files with the same name in 2026?

Practicing

Last night at the Vancouver Kafka meetup.

Finally, it's here! Time for an upgrade.

✍🏻 Interesting Links in the Data & AI World, February 2026 Links from @gunnarmorling.dev, @nbuesing.bsky.social, @ssp.sh, @sap1ens.com, @neelesh.bsky.social, @rstephens.me, @justincormack.bsky.social, @sogrady.org, @simonwillison.net, @cassidoo.co, and more! 🔗 rmoff.net/2026/02/27/i...

At Vancouver Systems meetup

Can a Rust project be 3x faster than a popular Java project right out of the box? I’m a big fan of RBIR: Rewrite Bigdata in Rust. So when @supermetal-inc.bsky.social recently launched Kafka sink support, I decided to take it for a spin and compare its performance with Debezium and Flink CDC.

Dream Theater not getting a Grammy in 2026 tells you everything you need to know about the recognition of the Rock/Metal bands in Grammys.

*sigh*

Engineers don't watch this talk enough and it shows (it's not about Python btw) www.youtube.com/watch?v=RZ4S...

@goldskyio.bsky.social × @supermetal-inc.bsky.social 30 TB. 18.9B rows. Seven figures saved. Supermetal handled the Postgres → Postgres migration so Goldsky could keep shipping. www.supermetal.io/customers/go...

Claude understands me.

🎄Interesting Links in the data world for December 2025 is here! There's some awesomely excellent content that you'll not want to miss. Grab a mince pie and dive in :) 🔗 rmoff.net/2025/12/16/i...

Does anyone have experience with github.com/plv8/plv8? I'm curious about performance.

Maybe, just maybe, having code review as the bottleneck is a... good thing?

Arrow + JavaScript WASM is quite easy now: github.com/uwdata/flech...

Do you like databases? Do you want to hear two database professors rant about them? Do you need one of those professors to have a Turing Award for databases? If yes, then join Mike Stonebraker and I next Wed Dec 10 @ 1:00pm EST for database hot takes: www.dbos.dev/webcast-2025...

I guess it's that time of the year

I've seen a lot of criticism directed towards Apache Flink recently. Here's something I have to say: Flink is not going anywhere. www.streamingdata.tech/p/why-apache...

The only newsletter I actually pay for. Highly recommend.

New post coming soon

Has anyone tried github.com/uwheel/uwheel?

The Advanced Apache Flink Bootcamp is now open for registration! The first cohort is scheduled for January 21st - 22nd, 2026. #ApacheFlink

Crazy stuff happening outside my programming bubble www.youtube.com/watch?v=HkgV...

🚀 Launching Supermetal — data replication that just works. Sync databases to warehouses in real-time or batch — no Kafka, no JVM, no Debezium. Built in Rust & Apache Arrow. Try it → trial.supermetal.io Launch post → supermetal.io/blog/launch

If you are in Vancouver on November 19th, you may want to check this Apache Kafka meetup! www.meetup.com/vancouver-ka...

My regular post-Current report is live: www.streamingdata.tech/p/current-ne...

Great conversation with Jan on all things Flink and data streaming youtu.be/X6Ukpi2p4y4