The Security Feature That’s Actually About Speed Here's the thing nobody's saying about Fabric Eventstream's new Custom CA and mTLS support: it isn't really a security feature. Or rather, it is, but the teams who'll benefit most aren't security teams. They're Spark engineers who've been running…

What the March 2026 Fabric Influencers Spotlight Tells Us About Where Spark Architecture Is Actually Heading The Fabric community has a habit of telling you the future before the product team ships it. You just have to know where to listen. Microsoft's March 2026 Fabric Influencers Spotlight…

Workspace Customer-Managed Keys for BYOK in Microsoft Fabric: Where Native Acceleration Pays Off and Where Fallbacks Bite Somewhere inside a regulated financial services firm, a platform engineer stared at two Fabric capacities on her screen. One ran Power BI semantic models encrypted with BYOK…

Your Fabric Spark Jobs Have Been Failing in Silence. That Just Changed. You schedule a notebook. You schedule a pipeline. You walk away. That's the deal. Set it and forget it. Except "forget it" has a dark side nobody warns you about. When a scheduled Spark job dies at 2 AM, it dies quiet. No…

What “Upgrade your Synapse pipelines to Microsoft Fabric with confidence (Preview)” actually means for Fabric Spark teams in production Preview posts are written to soothe. Production teams read them like incident reviewers. They want to know what moves, what stays off, and what still needs proof…

Fabric notebook resources in Git give Spark teams a real release boundary Most Spark notebook trouble is not caused by the notebook. It is caused by the little files loitering around it. A notebook runs fine in dev. In test, it quietly depends on a config file somebody copied by hand. In prod, a…

What the February 2026 gateway release really means for Fabric Spark teams Monthly gateway release posts are usually the corporate equivalent of dry toast. A version number appears. Power BI Desktop compatibility gets a polite bow. Then everyone goes back to moving data and arguing with refresh…

Bulk Import and Export Item Definitions Are the Fabric APIs Ops Teams Needed Most Fabric deployment pain is not dramatic. It is slow, dumb, and expensive in the worst way. Somebody asks you to move a workspace full of notebooks, pipelines, reports, and models. Then the afternoon disappears into…

The API layer that wasn’t supposed to matter The strangest platform announcements are usually the boring ones. Nobody throws a party for source control. Nobody leans back and says, "Hell yes, deployment pipelines," with a straight face. The applause goes to the flashy stuff: faster engines, new…

The Moment Your Spark Pipeline Stops Being a Pet There's a team, a data engineering squad at a mid-size financial services firm, who spent eighteen months building their Lakehouse architecture on Microsoft Fabric. They had notebooks. They had Spark job definitions. They had a deployment process…

DeltaFlow just changed the CDC conversation for Fabric Spark teams DeltaFlow preview in Fabric Eventstreams can turn raw Debezium CDC into analytics-ready streams for four supported database sources. For Spark teams, the opportunity is obvious and the caveat is too: test it now, but do not cut…

Operationalizing Fabric’s February 2026 feature drop: what actually matters for Spark teams Operationalizing Fabric's February 2026 feature drop: what actually matters for Spark teams Microsoft's monthly feature summaries have a familiar problem. They flatten every change into the same cheerful…

Operationalizing the semantic model permissions update for Fabric data agents Operationalizing the semantic model permissions update for Fabric data agents Permissions in data platforms have a remarkable talent for turning a two-minute job into a small municipal drama. You want one ordinary thing.…

ExtractLabel just changed how your Spark pipelines should handle unstructured data ExtractLabel just changed how your Spark pipelines should handle unstructured data Every data engineer eventually inherits the same cursed pipeline. Upstream sends you a blob of human text. Somewhere in that blob…

What “Recent data” in Fabric means for Spark teams when time is the real bottleneck Microsoft Fabric's February 2026 update shipped a small feature called Recent data in Dataflow Gen2. For Spark data engineering teams, it removes repeated navigation friction at the ingestion layer -- if you pair…

From CDC to Lakehouse: Making Fabric Eventstreams SQL Survive Contact with Production Spark From CDC to lakehouse: making Fabric Eventstreams SQL survive contact with production Spark Every data team eventually has the same bright idea: "Let's do CDC so everything is real time." What follows is…

Fabric Spark's Native Execution Engine: What Speeds Up, What Falls Back, and What to Watch

Open Mirroring + OneLake: Spark patterns that keep latency from eating your weekends Open Mirroring is deceptively simple to set up and tricky to run well with Spark at scale. Here are the architecture choices, anti-patterns, and validation checks that keep your pipelines from falling apart in…

What “Execute Power Query Programmatically” Means for Fabric Spark Teams What “Execute Power Query Programmatically” Means for Fabric Spark Teams Somewhere in a Fabric workspace right now, two teams are maintaining the same transformation twice. The BI team owns it in Power Query. The Spark team…

What the February 2026 Fabric Influencers Spotlight means for your Spark team What the February 2026 Fabric Influencers Spotlight means for your Spark team Microsoft published its February 2026 Fabric Influencers Spotlight last week. Twelve community posts. MVPs and Super Users. Most people skim…

Fabric Spark failure playbook: OneLake and mirroring under real production pressure A field-tested runbook for the failures that hide between Spark, OneLake, and mirroring in Microsoft Fabric: detection signals, triage sequences, and remediation tradeoffs from real production incidents.

The most boring technology announcement might be the most important one for your Fabric Spark team Microsoft's new ODBC Driver for Fabric Data Engineering looks like a checkbox feature. It isn't. Here's what it means for production Spark teams, the migration risks nobody's talking about, and a…

fabric-cicd Is Now Officially Supported — Here's Your Production Deployment Checklist

The Spark-to-Warehouse Connector in Fabric: What It Does, How It Breaks, and When to Use It

Fabric Spark billing just got clearer. Here's how to make the most of it.

From Demo to Production: ML-Enriched Power BI in Microsoft Fabric Microsoft's new end-to-end pattern for enriching Power BI reports with ML in Fabric looks clean in the demo. Here's the production migration checklist for Spark teams crossing the gap from notebook to ops.

Microsoft Fabric Warehouse + Spark: Interoperability Patterns That Actually Work

What SQL database in Fabric actually means for your Spark pipelines There is a particular kind of excitement that sweeps through data engineering teams when Microsoft announces a new database option. It is the same mixture of curiosity and low-grade dread you might feel upon learning that your…

Microsoft Fabric Table Maintenance Optimization: A Cross-Workload Survival Guide Your Delta tables are drowning. Thousands of tiny Parquet files pile up after every streaming microbatch. Power BI dashboards stall on cold-cache queries. SQL analytics endpoints grind through fragmented row groups.…

Optimizing Spark Performance with the Native Execution Engine (NEE) in Microsoft Fabric

The Best Thing That Ever Happened to Your Spark Pipeline Is a SQL Database

Monitoring Spark Jobs in Real Time in Microsoft Fabric If Spark performance work is surgery, monitoring is your live telemetry. Microsoft Fabric gives you multiple monitoring entry points for Spark workloads: Monitor hub for cross-item visibility, item Recent runs for focused context, and…

Running OpenClaw in Production: Reliability, Alerts, and Runbooks That Actually Work

Lakehouse Table Optimization: VACUUM, OPTIMIZE, and Z-ORDER If your Lakehouse tables are getting slower (or more expensive) over time, it’s often not "Spark is slow." It’s usually table layout drift: too many small files, suboptimal clustering, and old files piling up. In Fabric Lakehouse, the…

OneLake catalog in Microsoft Fabric: Explore, Govern, and Secure If your Fabric tenant has grown past "a handful of workspaces," the problem isn’t just storage or compute—it’s finding the right items, understanding what they are, and making governance actionable. That’s the motivation behind the…

Understanding Spark Execution in Microsoft Fabric Spark performance work is mostly execution work: understanding where the DAG splits into stages, where shuffles happen, and why a handful of tasks can dominate runtime. This post is a quick, practical refresher on the Spark execution model — with…

Fabric Spark Shuffle Tuning: AQE + partitions for Faster Joins Shuffles are where Spark jobs go to get expensive: a wide join or aggregation forces data to move across the network, materialize shuffle files, and often spill when memory pressure spikes. In Microsoft Fabric Spark workloads, the…

OneLake Shortcuts + Spark: Practical Patterns for a Single Virtual Lakehouse If you’ve adopted Microsoft Fabric, there’s a good chance you’re trying to reduce the number of ‘copies’ of data that exist just so different teams and engines can access it. OneLake shortcuts are one of the core…

When ‘Native Execution Engine’ Doesn’t Stick: Debugging Fabric Environment Deployments with fabric-cicd If you’re treating Microsoft Fabric workspaces as source-controlled assets, you’ve probably started leaning on code-first deployment tooling (either Fabric’s built-in Git integration or…

New OSS drop: Sparkwise (PyPI: sparkwise). Built by Santhosh Kumar Ravindran to help teams improve Fabric Spark price/perf with automated diagnostics + profiling. If you run Spark in Fabric, this will save you time and vCores.

Gil Gerard, Buck Rogers, and the Kind of Grief That Shows Up in December

Build Your Own Spark Job Doctor in Microsoft Fabric Microsoft Fabric simplifies Spark workload management but diagnosing performance issues remains challenging. This post introduces the "Job Doctor," an AI tool that analyzes Spark telemetry to identify problems like skew or excessive shuffles,…

Time to Automate: Why Sports Card Grading Needs an AI Revolution As I head to the National for the first time, this is a topic I have been thinking about for quite some time, and a recent video inspired me to put this together with help from ChatGPT’s o3 model doing deep research. Enjoy!…

Humans + Machines: From Co-Pilots to Convergence — A Friendly Response to Josh Caplan’s “Interview with AI” 1. Setting the Table Josh, I loved how you framed your conversation with ChatGPT-4o around three crisp horizons — 5, 25 and 100 years. It’s a structure that forces us to check our near-term…

Please - don't use whilst.

The Rise and Heartbreak of Antonio McDyess: A Superstar’s Path Cut Short

Enjoyed putting this together with ChatGPT's Deep Research capabilities - let me know if you like it!

The team worked a long time to make this a reality for folks - so excited it is finally here!! True serverless billing for Spark in Fabric! Introducing Autoscale Billing for Spark in Microsoft Fabric - blog.fabric.microsoft.com/en/blog/intr...