Our SIGMOD paper with our friends at Tsinghua + @wesmckinney.com + @pateljm.bsky.social on creating a next generation open-source data file format is out. F3 is a future-proof file format avoids the mistakes of Parquet. 📄 Paper: db.cs.cmu.edu/papers/2025/... 📁 Code: github.com/future-file-...

Why, hello there! 🪟 Thanks @rbn.bsky.social :D

In the last week I probably explained Rateless Set Reconciliation to a dozen other scientists. What an amazing paper and result, and already one year old.

What's a good discord server for people building things in the AI/LLM space?

Interesting workshop and a lovely community, would very much recommend!

Hi everyone! I'm a co-founder @dbos.dev, where we're building a serverless platform for highly reliable applications. I love conversations about databases, distributed systems, and anything technical. Thanks @qianli.dev for introducing me to Bluesky, and looking forward to meeting people here! 🦋

NULL BITMAP: in honour of DSQL's release last week I wrote about what the point of having extremely precise trustworthy clocks is buttondown.com/jaffray/arch...

CRDTs fit a system model that has hardly any overlap with datacenter-based applications. If I'm wrong, please point to datacenter-based apps which have benefited from the application of CRDTs

New blog post on the fun new hardware advancements which databases can leverage for great gains, and why the cloud means it doesn't matter that they exist. 🫠 transactional.blog/b...

ACM Symposium on Cloud Computing (SoCC) is next week! Lots of great papers on serverless, cloud, and AI/ML systems. I appreciate the push on diversity: both keynote speakers are women, plus a women in systems meetup! (This was my first time as a program committee member🤗) acmsocc.org/2024/index.h...

We've now published a blog about how LinkedIn runs large stateful apps with a generic operator on Kubernetes. www.linkedin.com/blog/enginee... Don’t forget to come to our #KubeCon session on Thursday.

I made a substantial update to my "messaging and eventing resources" page, adding the CNCF xRegistry project and related subprojects, pending proposals for Apache Avro, a section on Microsoft Fabric RTI, and a new talk recording. github.com/clemensv/mes...

Spent some time trying out #dotnet Aspire last night. Definitely impressed so far. I was surprised how easy it was to pull in non dotnet workloads too.

Distributed Systems & #DataBS folks, please welcome @lalithsuresh.bsky.social! Lalith authored my favorite paper on group membership, RAPID. Instead of building strongly consistent membership atop weak (eg, Raft + SWIM), RAPID delivers consistency from the get-go www.usenix.org/system/files...

CASPaxos is a Paxos variant which implements a linearizable (compare and set) register without needing logs. It's the simplest practical consensus algorithm, IMO. I wrote more about it here: reubenbond.github.io/posts/caspaxos

Over communicate if you’re remote. And map your communication style and message to the audience.

Scan RAM without trashing your CPU caches by leveraging non-temporal instructions, Memory Type Range Registers, or cache line flushing. Obscure, but perhaps useful for minimizing the impact of garbage collectors / LSM compaction. dl.acm.org/doi/pdf/10.1...

Hello friends! I'm so excited to join Bluesky here -- it feels like a cleaner and safer social platform. The first thing I did was to follow this instruction to set my domain as my handle. It's super simple! bsky.social/about/blog/4...

Is it end of the road for RocksDB in stream processing? Disaggregated state is the clearly superior architecture, with @responsive.dev investing heavily in SlateDB.io while Flink 2.0 has forked RocksDB. Here's why we've bet on SlateDB for Kafka Streams: www.responsive.dev/blog/why-sla...

Disaggregated storage pushes us to reexamine past ideas under a new light. Phil Bernstein gives middle-tier distributed transactions another look in this paper sites.computer.org/debull/A19ju.... What other examples come to mind?

Databases built on disaggregated storage (blobs) leverage reconfiguration rather than quorums. You don't need Paxos in your database if the storage layer guarantees linearizability and durability already.

Talking about Database Internals, I want to let y’all know that I just released another internals episode with the one and only @gwenshap.bsky.social If you want to know why Nile Postgres was created and how it supports multitenant SaaS, tune in NOW youtu.be/WZ7t0vqGKoo

Data Replication Design Spectrum by @alexmillerdb.bsky.social (transactional.blog/blog/2024-da...) pairs well with Categorizing How Distributed Databases Utilize Consensus Algorithms by Adam Prout (medium.com/@adamprout/c...). Both deserve a read if you are interested in distributed databases.

Orleans can automatically migrate frequently communicating grains to the same hosts, reducing network calls. Here's an app running on a real cluster showing grains (small circles) being migrated between hosts to eliminate network calls (red lines).

Stateful services: low latency, efficiency, scalability - pick three - Atul Adya (Databricks): hpts.ws/papers/2024/... Atul previously built Slicer at Google. Dicer is an evolution of that for Databricks. This presentation at HPTS resonated strongly with me - if only we had a recording

Papers would be better if they omitted part where they rub an LLM on the problem. I see two reasons why it's done: for novelty and to satisfy the authors' bosses

Orleans is such a backend secret-sauce 🔥 Awesome work !

A new feature in Orleans enables it to seamlessly balance already-active grains across your cluster as it scales or rolls. In this demo, 100K grains are activated on a single node. The cluster is scaled to 10 nodes and the grains are gradually spread across them until everything is balanced.

Around 70% of traffic in Azure is RDMA. Azure Storage is powered by RDMA between clients and frontends and within the clusters to improve I/O perf & save CPU

Language is both liberating and constraining. It lets us convey ideas but it also shapes what we can think. We can fumble our way into inventing new bits of language by giving names to concepts, allowing us to think new thoughts and express them. Naming things is difficult.

Lots of accounts here are mass following, presumably using a script. Does BlueSky want to prevent that kind of thing?

I'm Reuben, I work at Microsoft on the .NET team, building Orleans. I'm interested in distributed systems, databases, and making cloud app dev accessible to all developers