Rohan Desai
@rohanpd.bsky.social
📤 2095
📥 165
📝 3
Co-Founder of
@responsive.dev
. Views are my own.
reposted by
Rohan Desai
Apurva Mehta
11 months ago
This will be the foundation for being able to branch
@kafkastreams.bsky.social
apps to support seamless blue/green deploys. It also allows you to branch a previous version of the state to debug an issue from the past. Powerful stuff coming to the world of stream processing!
add a skeleton here at some point
1
13
4
reposted by
Rohan Desai
Sophie Blee-Goldman
about 1 year ago
For any Kafka Streams fans out there who want to know what it's like scaling an EOS application up to multiple TB of state at ~10k+ records/s with minimal downtime -- make sure to catch the Q&A we're hosting tomorrow with Metronome. Join us Dec 12th at 9:30am PST!
www.linkedin.com/events/72647...
loading . . .
Q&A: Scaling Event Driven Architectures with Metronome | LinkedIn
We're hosting a live Q&A session with two engineers who have seen it all: - Casey Crites, a founding engineer at Metronome, implemented and scaled the Kafka pipeline at Metronome from zero to where i...
https://www.linkedin.com/events/7264786447281336320/
0
10
4
reposted by
Rohan Desai
Apurva Mehta
about 1 year ago
This was a fun conversation with
@thegeeknarrator.bsky.social
and
@yingjunwu.bsky.social
about the world of SQL stream processing on one side and event driven apps on the other. Thanks for having me on the show, Kaivalya!
add a skeleton here at some point
1
7
3
reposted by
Rohan Desai
Apurva Mehta
about 1 year ago
It's said that Silicon Valley is special because the density of smart and motivated people leads to chance encounters that don't happen elsewhere. I can attest to that. Here's how a coffee resulted in
@responsive.dev
building a database optimized for stream processing in 8 months. (1/n)
1
23
8
reposted by
Rohan Desai
Apurva Mehta
about 1 year ago
From one of
@responsive.dev
's customers: "will report back .. when we start to change ttl lengths, which we're currently set up to do through launchdarkly based on the client and environment...pretty slick." They are changing the TTL of rows in their Kafka Streams state stores dynamically 🤩
0
7
2
reposted by
Rohan Desai
Apurva Mehta
about 1 year ago
Welcome
@ableegoldman.bsky.social
to BlueSky! Kafka Streams extraordinaire!
1
12
5
reposted by
Rohan Desai
Apurva Mehta
about 1 year ago
Some problems are impossible to solve without stream processing: for instance, did you know that
metronome.com
leverages Kafka and Kafka Streams to deliver real time billing features like spend limits at scale? (1/3)
1
7
5
reposted by
Rohan Desai
Almog
about 1 year ago
Coming from Kafka land, I’m always surprised at how “time” is an afterthought in many other databases. Hoping SlateDB gets it right: 1) have only one clock, 2) let users specify the clock, 3) enforce monotonic clocks, 4) use seq numbers (not time) for txns. Are we missing anything?
4
13
2
reposted by
Rohan Desai
Diptanu Choudhury
about 1 year ago
Looking forward to this! We just started using SlateDB in
@tensorlake.bsky.social
’s compute engine, looking forward to not needing dynamodb for CAS. One less moving part!
@chris.blue
when is the new release coming? :)
add a skeleton here at some point
1
9
1
reposted by
Rohan Desai
Chris
about 1 year ago
And it's done. Thank you so much
@benesch.bsky.social
for doing this. Now SlateDB will be able to have CAS for all major object stores. 🔥 It just works.
loading . . .
object_store: Conditional put and rename_if_not_exist on S3 · Issue #6285 · apache/arrow-rs
Is your feature request related to a problem or challenge? Please describe what you are trying to do. This is now supported on S3: https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3-condi...
https://github.com/apache/arrow-rs/issues/6285
2
17
2
reposted by
Rohan Desai
Chris
about 1 year ago
The new stack just put up a post on SlateDB!
loading . . .
SlateDB: 'Bottomless' Databases Built on Cloud Object Stores
SlateDB can dramatically cut costs of running a key-value store in the cloud, as long as users don't mind a bit of latency.
https://thenewstack.io/slatedb-bottomless-databases-built-on-cloud-object-stores/
4
42
6
reposted by
Rohan Desai
Chris
about 1 year ago
👋
@vigneshc.bsky.social
! All, Vinesh is a maintainer and committer for
slatedb.io
. He implemented the manifest and has been working on
fizzbee.io
proofs with JP. In his spare time, he’s a manager for Azure’s streaming team. 😁
0
18
3
reposted by
Rohan Desai
Kaivalya Apte
about 1 year ago
I am soon interviewing
@apurvamehta.com
from
@responsive.dev
and Yingjun Wu from
@risingwave.bsky.social
on all things stream processing. This is going to be an amazing episode. Feel free to ask any questions related to Stream processing and I will add the most interesting ones. Shoot!
6
17
7
reposted by
Rohan Desai
Apurva Mehta
about 1 year ago
Is it end of the road for RocksDB in stream processing? Disaggregated state is the clearly superior architecture, with
@responsive.dev
investing heavily in
SlateDB.io
while Flink 2.0 has forked RocksDB. Here's why we've bet on SlateDB for Kafka Streams:
www.responsive.dev/blog/why-sla...
loading . . .
If not RocksDB, then what? Why SlateDB is the right choice for Stream Processing.
Why we think SlateDB is the right foundation for a state storage service for Kafka Streams.
https://www.responsive.dev/blog/why-slatedb-for-kafka-streams
1
18
8
reposted by
Rohan Desai
Chris
about 1 year ago
This is gonna be cool... "The writer will also support initializing a new database from an existing checkpoint. This allows users to 'fork' an instance of SlateDb, allowing it to access all the original db’s data from the checkpoint, but isolate writes to a new db instance."
loading . . .
RFC for supporting checkpoints/clones by rodesai · Pull Request #278 · slatedb/slatedb
A cloud native embedded storage engine built on object storage. - RFC for supporting checkpoints/clones by rodesai · Pull Request #278 · slatedb/slatedb
https://github.com/slatedb/slatedb/pull/278/files
3
16
3
you reached the end!!
feeds!
log in