New from Materialize: Cloud M.1 Clusters Run 3x larger workloads with the same low latency and predictable performance—thanks to intelligent data spilling and expanded capacity. Learn more: bit.ly/3L12oH2

Not all operational data platforms are built alike. We break down the trade-offs between Materialize and Palantir Foundry in a new white paper. 📖 bit.ly/46LTjsO

Vector databases need fresh context to be useful. The challenge: keeping attributes up to date without burning compute or building brittle pipelines. Materialize fixes this with incremental updates, giving you faster, cheaper, fresher vector search. bit.ly/3KddzMs

Welcome Frank McSherry @frankmcsherry.bsky.social to Sync Conf 2025. Pioneer of sync technology, inventor of Differential Dataflow, and founder of @materialize.com, Frank will trace the evolution of sync and stream processing.

We’ve released a major improvement to our memory spilling infrastructure: Materialize now uses swap to scale SQL workloads beyond RAM. ✅ Faster hydration ✅ Efficient memory utilization ✅ Bigger workloads supported Full post from antiguru.bsky.social → bit.ly/46EF2iJ

I wrote about the projects done at Materialize’s recent hackathon. Many very cool projects, and also one that I worked on; take a read! materialize.com/blog/spring_...

At our last on-site, the Materialize R&D team held a hackathon. 8 projects. 1.5 days. Highlights: – SQL tutorial game – WASM UDFs – API endpoints from views – S3 as a consensus layer One shipped already. Others might next. Read the full recap → bit.ly/4lo4YmR

New white paper: Materialize vs ClickHouse How to choose the right tool for real-time vs historical analytics — and why modern data platforms often need both. Dive into architectural comparisons, use cases, and case studies: bit.ly/412qx5b #DataInfrastructure #ClickHouse #Materialize #AIDataLayers

Imagine… A live data layer built for apps *and* agents That incrementally maintains views at the scale of >1M updates per second While maintaining up-to-the-second freshness With query response times in the single-digit milliseconds

Waiting for CI hurts. In July, we cut our runtime by up to 86%. From 23+ min builds to under 2 min, and full runs in as little as 7 min. Caching, parallelization, smarter builds, and a bit of [libeatmydata] magic. How we did it 🔗 bit.ly/45yoOWM

AI agents need more than stale snapshots — they need a real-time model of the world. Materialize powers digital twins: always-fresh, SQL-accessible representations of your business. How to build them: bit.ly/46H97i7

Materialize can "push down" the filters in your query to its storage layer to fetch less data — and thanks to a few cool static analysis tricks, this works for more queries than you might expect. To see how it works, check out the blog: bit.ly/475FBCL

Want live analytics on Bluesky itself? Pipe the public firehose into Materialize with a tiny JS script, then explore trends in SQL. Full walkthrough by @frankmcsherry.bsky.social → bit.ly/46OOwsa

We have a new blog post up at @materialize.com about analyzing the Bluesky firehose (Jetstream, really) through Materialize. You can grab a copy of the community edition of MZ and follow along, or invent your own ways of looking at the data, live! materialize.com/blog/analyzi...

Untangling control vs. data paths :point_right: Bigger SELECT results, smaller bottlenecks. Materialize now streams large query outputs out-of-band, so coordination stays snappy while data flies. Dive into the architecture shift and what it unlocks next → bit.ly/3Ub6GwI

SponsorCX went from 90-minute batch updates to ~1-second freshness by pointing Materialize at Postgres. No streaming specialists—just SQL. Real-time reporting shipped the same day. Check out the full story: bit.ly/4lM1k6Y

Neo Financial now serves real-time features that are fresh and fast while saving 80 % on infra. All SQL, no cluster babysitting. Case study → bit.ly/4lIP8E3

I refreshed a blog post draft on streaming the Bluesky firehose through @materialize.com. Some experience tidied up the examples, made things a bit more efficient, and told a different story (now with less cloture). More in the near future, as we put a front end on it! github.com/frankmcsherr...

Flink vs Materialize isn’t apples-to-apples. Flink is a stream processor with external dependencies. Materialize is a unified platform: ingest, transform, and serve real-time data in SQL. 💡 50% faster deploys 💰 45% lower cost 📖 Read the guide: bit.ly/4eBNMc0

LLM agents that act need data that reacts. If your data layer can’t reflect the consequences of an agent’s action in real time, it’s not just inefficient—it can lead to disaster. 🧠 Smarter agents need smarter data. bit.ly/4lz4hro #AI #DigitalTwins #LLM #Materialize

Materialize 25.2 is here! New features include live freshness reports for all your views, 2.5x faster data product deployment times, and native SQL Server support. See how these updates can help streamline your operations: bit.ly/44i2hg2

Big news: Materialize now connects directly to SQL Server. We ingest CDC, maintain real-time views of your logic, and eliminate the pain of: - Slow OLTP queries - Stale dashboards - Brittle pipelines Just SQL. Just correct. Just live. 🔗 bit.ly/4mKbk1S

Just a normal day at work where a co-worker discovers a memory bug in Rust

The Materialize engineering team uncovered a rare concurrency bug 🪲in Rust’s 🦀 unbounded channels that could lead to double-free memory errors. After thorough debugging and working with the Rust and crossbeam communities, the fix is now part of @rust-lang.org 1.87.0. 🔗 bit.ly/3Fan1Om

AI is pushing data infrastructure to its limits. MCP gives agents access to services—including databases—but most systems can’t handle the load. Materialize’s MCP server turns live data products into tools agents can use—without crushing your systems or overwhelming your team. bit.ly/4jYBrQU

Agents generate more data and place more demand on systems than ever before—and standards like MCP will only accelerate this trend. Learn how Delphi is rethinking how they build data-intensive applications—from the db to the UI: bit.ly/42BZdf9

I just put up a video of me using @materialize.com to ingest and analyze live Bluesky data. Starts with a Javascript bridge to get the data in, and then builds up various live views using SQL. Caveat: it's a live demo with live data. Eek! www.youtube.com/watch?v=T6cY...

at the rustnyc meetup and the vibe is electric! thanks to @materialize.com for hosting. (they just released their self-managed version today!) #rust #rustlang

I'm very excited to share that @materialize.com now has a Self-Managed option, as well as a Community Edition license that lets you play around with the exact same bits but for free. materialize.com/blog/materia...

To keep up with the demands of AI projects your teams need to transform, deliver, and act on fast changing data no matter where it resides. Today, we’re making that possible by moving Self-Managed @materialize.com out of beta and introducing Materialize Community Edition (CE).

Materialize for Everyone: Now Self-Managed & Free 💡 Self-Managed – Deploy in your own cloud, full control. 💡 Community Edition – Production-grade Materialize, free forever. Real-time data is now open to all. Learn more: bit.ly/43Bg8iQ

🚀 Debugging Materialize with Materialize 🚀 Materialize now exposes source maps, mapping runtime metrics to high-level operators. ✅ Pinpoint bottlenecks ✅ Get EXPLAIN ANALYZE insights ✅ Tune TopK queries efficiently Deep dive: bit.ly/3Eq2l4g #databases #queryoptimization #Materialize

How does your data system handle constant change? Materialize delivers real-time consistency for evolving, transactional data sources. @frankmcsherry.bsky.social explains how virtual time aligns data timelines for guaranteed accuracy. Learn more: bit.ly/40SZVCX

Materialize introduces Replica Expiration! Our latest optimization cuts memory use by up to 50% while maintaining performance and accuracy. 🔗 bit.ly/4aDkdEL #Materialize #StreamingSQL #DataOptimization

We just landed the first use of the columnar crate in Materialize! Stay tuned for more. github.com/MaterializeI...

@frankmcsherry.bsky.social at #rustnyc: the main problem with my presentation is that I'm too excited and I'll tell you the story out of order. Let's gooo

Big thanks to the #rustnyc audience; you all were great! Patience, smiles, and thoughtful questions all around. This happens regularly, so if you are in the NYC area and interested in Rust, keep tabs and show up in the future: www.meetup.com/rust-nyc/

Want AI that acts, not just informs? 🚀 Discover how Materialize delivers real-time, consistent structured data to supercharge RAG systems. Say goodbye to stale insights and hello to AI that drives action. Read more: bit.ly/4anPBHl #AI #RAG #RealTimeData

Simplifying microservices data integration—without compromising performance or scalability—is possible. Check out our new blog on shared databases, materialized views, and redefining microservices architecture with Materialize. Learn more: bit.ly/4jhHG2g

@frankmcsherry.bsky.social is revisiting his first Rust crate—columnar—and sharing how he’s updated it to solve complex memory layout challenges with modern, safe Rust. When: Jan 21 | 6:30 PM (🍕), 7:30 PM (🎤) Where: Materialize HQ Love Rust or clever systems design? Join us! bit.ly/3PCNype

I wrote a bit about efficient maintenance of temporal filters in @materialize.com. By periodically restarting replicas, we can limit resource requirements (RAM and CPU), all without any other user-visible impact. materialize.com/blog/replica...

Self-Managed Materialize is here—early access is open. Run Materialize in your private cloud or wherever your infrastructure demands. Maintain real-time capabilities while keeping full control over your data. Learn more about how it works: bit.ly/3ZXVnM3

The future of autonomous systems isn’t just smarter agents—it’s intelligent orchestration. 🌐 Learn how Materialize helps AI agents collaborate in real-time, share fresh data, and scale intelligently. Read the blog: bit.ly/4fk1qyT

turns out Materialize can help out with microservice architectures... I wrote about it here.

Are microservices worth the hype? They promise flexibility but bring data consistency issues and hidden overhead. Our blog dives into the challenges and a better way forward with real-time data: 🔗 bit.ly/3VCuzyn #Microservices #RealTimeData

S3 (Iceberg) Tables is everything I dreamt of, and more. I blogged some long-form thoughts: meltware.com/2024/12/04/s... I think we're about to see an explosion of data tools (@materialize.com, @clickhouse.com, @duckdb.org, et al.) learn to write Iceberg tables via S3 table buckets. #databs

materialized views and it's the same but there's incremental updates so it's not Stop by booth #1761 to hear how Materialize is re:Inventing the Operational Data Store. #AWSreInvent #Materialize #RealTimeData #GenerativeAI #Brat #materializedviews #incrementalupdates

Now for more videos about recursive SQL! This time, an imo natural combination of sudoku solving and mutual recursion. What we'd write in pencil (options) and what we'd write in pet (certain) co-develop, leading (in this case) to the right answer: youtu.be/VTjkwtsJK7c

Sometimes it's easier to mitigate a problem than solve it: a feature I recently implemented reduces the memory and CPU @materialize.com requires to maintain future updates. Some details why solving the problem is hard and the mitigation easy-ish. materialize.com/changelog/20...