If I ever design a system badly… I won’t admit.. But there will be signs. - Every software engineer ever.

1/12. I'm excited to share our latest technical blog post on ParadeDB. After a brief hiatus focused on transforming ParadeDB into an enterprise-ready database, expect to hear a lot more from us. Today's post: How ParadeDB built an LSM on top of Postgres block storage. 🧵

How do LSM trees work? Finally created a light board video. I hope you like it 🙏🏻 LSM Trees: How does it all work? youtu.be/CizxYuq-z6E

Another advantage to designing systems as state machines: automated testing and even verification becomes dramatically easier.

What is State Machine thinking and how do I use it (practically)? State machine thinking is one of the best ways to build reliable software. The idea is to break down complex systems into discrete states and explicit transitions between them.

What is First principles thinking and how do I use it (practically) ?👇 First-principles thinking is one of the best ways to tackle complex problems. The idea is to break down complex problems into primitive elements and then reassemble the problem from scratch.

Small enough service is a micro-service. Large enough language model is a Large Language Model. Naming is a NP-hard problem.

Here is a sketch of how a solid observability platform built on top of object storage would look like this 👇 And guess what, Parseable is built exactly like that. All the good stuff that makes a system fast, reliable and highly scalable. New episode 🚀 youtu.be/MugLlrf5E_g

Diskless Kafka? But why? Why is it important for Kafka ecosystem ? How does it compare with existing revolutionary technologies? Blogged a comparison between Diskless Kafka, WarpStream and Confluent Freight. www.geeknarrator.com/blog/diskles...

Diskless Kafka? But why? Why is it important for Kafka ecosystem ? How does it compare with existing revolutionary technologies? Blogged a comparison between Diskless Kafka, WarpStream and Confluent Freight. www.geeknarrator.com/blog/diskles...

Blogged ✍🏻 Schema Driven Development with Buf www.geeknarrator.com/blog/buf-sch...

DUPLICATE events and painfully SLOW joins are a real and painful problems in the world of stream processing and realtime analytics. But why there hasn't been a simple, easy and fast solution for these problems that just works with the hassle?

DUPLICATE events and painfully SLOW joins are a real and painful problems in the world of stream processing and realtime analytics. But why there hasn't been a simple, easy and fast solution for these problems that just works with the hassle?

BLOGGED 🎉 - Diskless Kafka KIP-1150 Wrote a detailed blog on why I think diskless Kafka is going to be HUGE. Do check it out here: www.geeknarrator.com/blog/diskless-kafka-kip-1150 Thanks Aiven for sponsoring the blog.

There is a lot of innovation happening in the Databases world BUT one area where there is a lot of opportunities is Query optimisers. I recently learnt about query optimisers and the amazing work CMU db group is doing in this space with their new query optimiser “Optd”.

Are there Senior/Staff+ Engineers on my timeline who want to work on the intersection of #Databases, #Caches, #Infra, Kubernetes/Operators and #Scaling #Algorithms (think time series, predictive scaling etc)? If so, please drop me a DM. Our small but mighty team at ClickHouse is growing!

If someone says “I am a perfectionist”… they are either lying OR mean they are not going to get/let things done on time. The only realistic way is to be above a certain quality bar and keep the progress iterative and incremental. Get things done fast.

🎵 Sweet dreams are made of C++ 🎵 When something isn't working, sometimes you just need to rebuild it from scratch. That's what our Founding Panda, @emaxerrno.bsky.social, did when building Redpanda 🚀 Want to hear his story? Listen to Alex's chat with @thegeeknarrator.bsky.social in this episode 🎧

Thanks to @thegeeknarrator.bsky.social for having me! This was a fun podcast. 🚀

Listen to @thegeeknarrator.bsky.social podcast! @feldera.bsky.social CEO, @lalithsuresh.bsky.social, discusses Feldera's approach to incremental view maintenance and the foundational breakthrough -- DBSP -- that powers it, engine internals, storage, handling late events. youtu.be/GpcIoABwF60?...

Listening to the DuckDB Internals with Mark Raasveldt (@markraasveldt.bsky.social) episode of The Geek Narrator podcast (@thegeeknarrator.bsky.social): www.youtube.com/watch?v=f9Ql...

Just imagine writing a mature and massive system like #Kafka from scratch in C++. And on top, challenge yourself to make it 10x faster in tail latencies and way more cost effective.

Just imagine writing a mature and massive system like #Kafka from scratch in C++. And on top, challenge yourself to make it 10x faster in tail latencies and way more cost effective.

In this episode of @thegeeknarrator.bsky.social The GeekNarrator, Richard Artoul and Ryan Worl discuss: 🚀 WarpStream architecture. ✅ Kafka ordering guarantees. 🗜️ Buffering + compaction. 💸 WarpStream flexibility + cost efficiency.

Learn the JVM internals 👇 1) The JVM Internals - Basics (Episode 1) youtu.be/Yg69qM0iZa4 2) JVM Internals: Heaps, Stacks, Program Counters and more youtu.be/MCTTg8VPaBY 3) JVM Internals - The Class file format youtu.be/15sz5yvQ-Po

We are focused on high-quality talks and expert speakers. Talks about engineering, not product pitches. Currently looking for speakers and venues that can host >50 people!

Do one thing with best quality. > Do less things with good quality. > Do many things with subpar quality. > Do nothing. > Nothing.

The JVM internals series - #5 is out.. This episode lays out a good foundation for us to go into the internals of each of the amazing GCs and their tradeoffs. Become a member to access the upcoming internals episode.. Meanwhile watch this video (free access until tomorrow). youtu.be/p1Yamdly0QE

New episode "Patterns of Distributed Systems" is up now.. Experience based, Actionable insights is what you can expect from this episode. Enjoy: youtu.be/HnyKJq8oR0M Image taken from: martinfowler.com/articles/pat...

This is HUGE! Congratulations to the team.

Totally agreed!! I really enjoyed our podcast interview with @thegeeknarrator.bsky.social — thoroughly prepared with informative insights and questions.

Wow thanks a lot @chris.blue 🙏🏻

I had a great time talking to Kaivalya Apte about Aurora DSQL on the Geek Narrator podcast. Watch our episode here: www.youtube.com/watch?v=ONkf... We cover the design of Aurora DSQL, interesting aspects of the design, PostgreSQL, Firecracker, and trade-offs of database systems. A lot in an hour!

Today I released the next episode on The GeekNarrator, with @marcbrooker.bsky.social on AWS Aurora DSQL internals. This is filled with amazing insights. Don't miss this one! youtu.be/ONkfXvPSSpc

Crunching too many numbers inside your Postgres database? Is it crawling along while doing analytics? Use pg_duckdb to power the elephant with the duck. Learn more from contributor @jeltef.nl in this interview on a podcast by @thegeeknarrator.bsky.social. www.youtube.com/watch?v=Qn9_...

What happens when you create a "new" Object in Java? What are the challenges in a multithreaded environment? Does GC play any role in object creation? New video will answer these questions for you. Get access here www.youtube.com/channel/UC_m... Link to the video: youtu.be/qfcnB8kciCE

The best #SystemDesign interview question is something that is relevant to your business domain and helps you understand how well the candidate suits the requirements and the team setup.

How do you build reliable applications faster? By taking the standard reliability patterns and applying them to your applications. Just released first episode of 2025 with Peter Kraft and Qian Li. youtu.be/F0nXTAaXYaY

We had a wonderful time chatting with Kaivalya about @dbos.dev, durable execution, and serverless on The Geek Narrator podcast! I was impressed by how much thought and preparation went into Kaivalya's insightful questions, which made for an engaging discussion. Check it out 🤗

How do you build reliable applications faster? By taking the standard reliability patterns and applying them to your applications. Just released first episode of 2025 with Peter Kraft and Qian Li. youtu.be/F0nXTAaXYaY

This is an excellent summary. Reminds me in some ways of star tree indexes. Also feels like an LSM B-tree mashup. Interesting tradeoffs. Who's building this on object storage? 😅

I started my 2025 reading white-paper(s) on Bε-trees. A thread that captures everything you need to know about them and more.. Like, repost and share..👇

Deep down, everyone knows, “I will start doing XYZ from Jan 1st/New Year” is just an excuse and they are going to fail at it, worse than they did, an year ago.

One of the Holiday projects completed. Was tougher than what I thought and easier than what my wife thought. Now movie time. One of my favourites.

A couple of more books and I am all set. 2025 is going to be amazing. Stronger fundamentals. More Deep work. Quality content. Along with podcasts, it will be filled with series on Distributed systems, JVM internals, Database internals and Stream processing.

Happy Holidays folks.. What are y’all reading?

I am going to interview @marcbrooker.bsky.social soon. Of course, we will talk about Aurora DSQL. But feel free to add specific topics/questions. Really excited for this one. Needs good preparation though.

Great podcast! Congratulations!

Feels unreal to get the silver button 🙏🏻♥️ Thanks everyone for the support. Thanks all my lovely guests who made The GeekNarrator what it is today. There can’t be a better way for me to wrap up 2024. www.youtube.com/c/TheGeekNar...

In the context of infrastructure as code, what qualifies as infrastructure? What properties should it have? Hot take: I don’t think just having a terraform provider makes it infrastructure.